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This  report  describes  a research  project  that  was  undertaken  to  gain 
a better  understanding  of  how  simulator  training  might  be  made  more  effec- 
tive in  the  U.S.  Air  Force.  This  effort  was  begun  when  the  author  was 
employed  by  the  Human  Resources  Research  Organization  (HumRRO) . The  effort 
was  completed  and  the  present  report  prepared  under  the  auspices  of  the 
Seville  Research  Corporation,  under  HumRRO  subcontract  number  SubE  77-04-05. 
Dr.  Caro  is  presently  a Senior  Scientist  with  Seville  Research  Corporation. 

The  project  built  upon  an  earlier  literature  review  by  the  author 
when  he  was  employed  by  HumRRO.  That  review  was  reported  in  "Some  Factors 
Influencing  Simulator  Training  Effectiveness,"  a paper  presented  at  the 
Third  Flight  Simulation  Symposium  of  the  Royal  Aeronautical  Society, 

London,  England,  in  April,  1976.  The  paper  also  was  published  as  HumRRO 
Professional  Paper  1-76,  August,  1976. 

The  research  reported  here  was  sponsored  by  the  Life  Sciences  Direc- 
torate, Air  Force  Office  of  Scientific  Research,  Air  Force  Systems  Command, 
under  Contract  No.  F44620-76-C-01 18 . Dr.  Alfred  R.  Fregly  was  the  Program 
Manager  for  AFOSR.  Dr.  Paul  W.  Caro  was  the  Principal  Investigator 
throughout  the  project,  both  while  with  HumRRO,  and  with  Seville  Research 
Corporation. 
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SUMMARY 


PURPOSE  AND  APPROACH 

A study  of  simulator  training  was  conducted  to  identify  factors  that 
influence  the  effectiveness  of  such  training  and  to  learn  how  simulator 
training  effectiveness  is  determined  by  operational  users  of  simulators  in 
the  U.  S.  Air  Force.  The  research  consisted  of  two  principal  activities: 
a survey  of  representative  Air  Force  simulator  training  programs,  and  a 
review  of  the  simulator  training  research  literature.  Both  activities 
concentrated  upon  the  identification  of  factors  influencing  simulator  train- 
ing effectiveness  and  the  methods  used  to  determine  simulator  training  effec- 
tiveness. During  the  survey  portion  of  the  research,  simulator  training 
activities  of  the  Aerospace  Defense  Command,  the  Military  Airlift  Command, 
the  Strategic  Air  Command,  and  the  Tactical  Air  Command  were  visited,  simulator 
training  programs  were  observed,  and  interviews  were  conducted  with  simulator 
instructors  and  trainees  as  well  as  with  personnel  responsible  for  the  develop- 
ment and  management  of  simulator  training.  Ten  combat  crew  training  and  con- 
tinuation training  activities  at  nine  Air  Force  Bases  were  included  in  the 
survey.  During  the  literature  review,  a variety  of  library  sources  were  con- 
sulted, with  a concentration  on  studies  of  training  effectiveness  and  evaluation 
procedures . 


FINDINGS 

Ten  simulator  training  effectiveness  study  designs  that  have  been  used  in 
studies  of  simulator  training  effectiveness  were  identified  and  described  in 
terms  of  simple  models.  The  descriptions  of  these  models  include  information 
concerning  the  relative  value  of  each  with  respect  to  the  relevance  and  ob- 
jectivity of  the  data  it  yields.  The  efforts  by  the  Air  Force  to  validate  the 
simulator  training  activities  surveyed  are  described  in  relation  to  these  ten 
study  design  models.  It  was  found  that  the  programs  surveyed  had  not  been 
subjected  to  formal  evaluation  studies  that  would  establish  their  training 
effectiveness  in  quantitative  terms.  In  those  instances  in  which  attempts  hod 
been  made  to  validate  simulator  training,  a tendency  was  noted  to  employ  study 
models  that  were  based  upon  subjective  opinions  rather  than  upon  objective 
data  collected  during  transfer  of  training  studies. 
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A number  of  suspected  or  potential  factors  influencing  simulator 
training  effectiveness  were  identified  during  the  project.  These  factors 
include  simulator  design  for  training,  simulator  visual  display  fidelity, 
simulator  platform  motion  system  fidelity,  simulator  handling  character- 
istics, simulator  training  program  features,  simulator  trainee  and  instruc- 
tor characteristics,  and  attitudes  and  expectations  toward  simulator  training. 
The  discussion  of  each  of  these  factors  reviews  relevant  literature  and  Air 
Force  simulator  and  training  system  design  features  and  training  practices. 

The  available  information  concerning  the  influence  of  these  factors 
upon  simulator  training  effectiveness  was  found  to  be  quite  limited.  Because 
of  the  absence  of  objective  studies  validating  Air  Force  simulator  training 
effectiveness,  the  Influence  of  factors  identified  during  the  survey  upon 
such  training  could  only  be  hypothesized.  Additionally,  definitive  data 
could  seldom  be  found  in  the  literature  reviewed  that  would  permit  the  quanti- 
fication of  the  influence  of  many  suspected  factors,  and  methodological  prob- 
lems made  it  difficult  to  generalize  conclusions  from  the  literature  to  the 
Air  Force  programs  surveyed.  Therefore,  the  information  presented  in  this 
report  regarding  influences  upon  simulator  training  effectiveness  in  some 
instances  is  suggestive  rather  than  conclusive. 

RECOMMENDATIONS 

Principal  recommendations  are  as  follows: 

— Increased  emphasis  should  be  placed  upon  validating  Air  Force  simulator 
training  activities,  employing  validation  study  design  models  that  emphasize 
objective  measurement  of  trainee  performance  in  operational  aircraft  against 
predetermined  performance  standards. 

— Research  should  be  undertaken  to  examine  simulator  design  considera- 
tions as  a function  of  specified  training  objectives  and  of  the  manner  in 
which  the  devices  are  to  be  used  to  achieve  those  objectives. 

— Research  should  be  undertaken  to  determine  the  cognitive  and  visual 
cues  essential  to  the  attainment  of  visual  training  objectives  and  to  find 
means  of  attaining  those  objectives  that  do  not  rely  exclusively  upon  extra- 
cockpit visual  simulation. 

— Research  should  be  undertaken  to  examine  separately  the  influences  of 
maneuver  and  disturbance  motion  cues,  with  particular  attention  to  an  analysis 
of  disturbance  motion  cues  in  relation  to  specific  training  objectives. 
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— Reviews  of  Air  Force  simulator  training  activities  should  he  con- 
ducted to  identify  areas  in  which  better  use  could  be  made  of  available 
information  in  the  areas  of  human  learning  and  performance. 

— Increased  emphasis  should  be  placed  upon  the  needs  of  individual 
trainees  in  the  development  and  administration  of  simulator  training  programs. 

— Research  should  be  undertaken  to  identify  the  instructor  skills  and 
techniques  needed  for  effective  and  efficient  simulator  training,  and 
training  programs  in  which  such  skills  and'  techniques  could  be  developed 
should  be  provided  all  simulator  instructors. 

— Existing  administrative  practices  related  to  simulator  training  should 
be  examined  to  assure  that  they  are  conducive  to  favorable  attitudes  toward 
simulator  training  and  to  the  effectiveness  of  that  training. 
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SOME  FACTORS  INFLUENCING  AIR  FORCE 
t 

SIMULATOR  TRAINING  EFFECTIVENESS 


I . INTRODUCTION 


BACKGROUND 

The  U.  S.  Air  Force  presently  is  one  of  tlie  major  users  of  flight 
simulators,  and  the  extent  of  its  use  of  these  devices  is  increasing. 

While  the  reasons  for  the  growing  role  of  simulators  in  aircrew  training 
are  manifold,  the  major  factors  to  which  these  changes  can  be  attributed 
include  the  increased  capability  of  simulators  resulting  from  recent 
engineering  innovations;  the  combined  effects  of  increased  costs  of  op- 
eration of  training  aircraft  and  reduced  Department  of  Defense  funding 
levels;  actual  as  well  as  potential  future  reductions  in  fuel  resources 
needed  to  support  aircraft  operations;  concerns  of  environmentalists  over 
the  possibly  deleterious  effects  of  aircraft  operations  upon  the  environ- 
ment; and  the  increased  safety  in  training  which  simulator  training  per- 
mits. The  present  trend  toward  more  extensive  use  of  flight  simulators 
to  meet  operational  training  requirements  is  unlikely  to  be  reversed 
because  of  the  impact  of  factors  such  as  those  identified  above,  as  well 
as  the  training  benefits  which  have  been  attributed  to  the  use  of  flight 
simulators  in  non-military  settings.^ 

As  simulator  use  continues  to  increase,  the  interest  of  the  Air 
Force  and  other  simulator  users  clearly  is  to  achieve  effective  training 
through  simulation  rather  than  merely  to  use  simulators  more  extensively. 
There  has  been  a tendency  in  the  past  among  some  simulator  training  mana- 
gers to  make  the  tacit  assumption  that  all  simulators  are  optimally  de- 
signed and  used,  so  simulator  training  effectiveness  is  a matter  primarily 
of  how  much  the  devices  are  used.  Such  is  not  the  case,  however.  It  is 
possible  to  use  any  simulator  extensively,  while  at  the  same  time  to  use 


^American  Airlines,  Inc.  Flight  Training  Academy.  Optimized  Flight 
Crew  Training,  A Step  Toward  Safer  Operations.  American  Airlines,  Inc., 
Fort  Worth,  TX,  1969. 
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it  effectively.  In  one  study,  for  example,  it  was  found  that  the  exten- 
sive use  of  a particular  device  added  cost,  but  no  training  value,  to  an 

already  expensive  pilot  training  program,  a finding  which  was  subsequently 

2 

attributed  largely  to  that  particular  device's  less  than  optimum  design. 
Another  study  demonstrated  that  the  training  value  of  a device  could  be 
increased  substantially  without  increasing  the  amount  of  device  training 
time  involved  when  needed  changes  were  made  in  the  manner  in  which  it  was 

3 

used.  Effective  simulator  training  is  critical  if  the  Air  Force  is  to  comply 

with  the  Department  of  Defense  planning  guidance  goal  of  25%  reduction  in 

the  use  of  aircraft  for  flight  training  by  1981  while  maintaining  the  present 

4 

quality  of  that  training. 

Because  of  its  extensive  present  and  future  reliance  upon  simulators 
in  meeting  aircrew  training  requirements,  the  Air  Force  has  a continuing 
interest  in  increasing  the  effectiveness  of  each  of  its  simulator  training 
programs.  Two  major  thrusts  characterize  that  interest:  (a)  the  develop- 

ment of  simulators  with  the  potential  capability  for  providing  major  portions 
of  the  training  required;  and  (b)  the  development  of  techniques  and  proce- 
dures for  simulator  use,  following  systematic,  analytic  procedures  such  as 
Instructional  Systems  Development  (ISD) , which  are  directly  responsive  to 
training  needs.  Enhancement  of  the  effectiveness  of  its  simulator  training 
is  a major  goal  of  the  Air  Force. 

^Isley,  R.  N. , Caro,  P.  W. , and  Jolley,  0.  B.  Evaluation  of  Synthetic  In- 
strument Flight  Training  in  the  Officer-Warrant  Officer  Rotary  Wing  Aviator 
Course . Technical  Report  68-14,  Human  Resources  Research  Organization, 
Alexandria,  VA,  1968. 

2 

Caro,  P.  W.  Equipment-Device  Task  Commonality  Analysis  and  Transfer  of 
Training.  Technical  Report  70-7,  Human  Resources  Research  Organization, 
Alexandria,  VA,  1970. 

3 

Caro,  P.  W. , Isley,  R.  N.,  and  Jolley,  0.  B.  Research  on  Synthetic  Training: 
Device  Evaluation  and  Training  Program  Development.  Technical  Report  73-20, 
Human  Resources  Research  Organization,  Alexandria,  VA,  1973. 

4 

Hearing  before  the  Subcommittee  on  Research  and  Development  of  the  Committee 
on  Armed  Services.  U.  S.  Senate,  94th  Congress,  Second  Session,  May  13,  1976. 
U.  S.  Government  Printing  Office,  Washington,  DC,  1976. 
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PURPOSE 


The  present  study  was  undertaken  in  an  attempt  to  gain  a better  under- 
standing of  how  this  training  effectiveness  goal  might  be  achieved.  Spe- 
cifically, the  primary  purpose  of  the  present  study  was  to  identify  the 
principal  factors  that  influence  the  effectiveness  of  Air  Force  simulator 
training  programs.  The  need  for  such  a study  is  based  upon  an  assumption 
that  some  simulator  designers  and  users  may  be  unaware  of  factors  that, 
if  properly  treated  or  managed,  would  markedly  enhance  the  value  and 
efficiency  of  simulator  training.  A likely  consequence  of  lack  of  aware- 
ness of  such  factors  is  that  simulator  training  effectiveness  will  suffer, 
e.g.,  skills  that  are  critical  to  safe  operation  of  an  aircraft  may  not  be 
developed,  aircraft  may  be  used  unnecessarily  for  training,  training  costs 
may  become  excessive,  or  simulator  training,  although  effective,  may  be 
conducted  in  an  inefficient  manner. 

In  order  to  identify  factors  that  Influence  Air  Force  simulator  train- 
ing effectiveness,  an  immediate  question  arises  as  to  the  relevant  criteria 
of  effectiveness.  Effectiveness  implies  that  simulator  training  has  some 
demonstrable  effect  upon  performance  of  the  trainees.  Ideally,  the  effect 
should  be  positive  in  direction  and  should  be  evident  either  in  terms  of 
improved  performance  in  an  aircraft  subsequent  to  simulator  training,  or  in 
terms  of  more  efficient  or  less  costly  attainment  of  a given  level  of  skill. 

The  scope  of  the  present  research  was  limited  to  the  collecting  of 
already  existing  information  concerning  simulator  training  from  Air  Force 
and  other  available  sources.  Because  of  this  limitation  in  scope,  it  was 
no”  Dossible  during  this  study  to  verify  experimentally  that  a particular 
factor  did  or  did  not  influence  Air  Force  simulator  training  effectiveness, 
except  to  the  extent  that  existing  experimental  or  other  data  could  serve  that 
purpose.  Therefore,  a secondary  objective  of  the  research  reported  here  was 
to  locate  existing  data  that  might  verify  the  influence  upon  simulator 
training  effectiveness  of  factors  identified  during  the  study.  A part  of 
this  secondary  objective  was  to  determine  the  manner  in  which  such  existing 
data  were  obtained  so  that  their  validity  and  reliability  could  be  assessed. 


APPROACH 


>,  Conduct  of  the  research  described  herein  was  directed  toward  two 

questions  underlying  its  primary  and  secondary  objectives:  (1)  Wliat  fac- 

tors influence  simulator  training  effectiveness?  and  (2)  How  was  effec- 
tiveness of  simulator  training  determined  in  those  activities  in  which 
these  factors  were  found  to  be  operative?  The  research  consisted  of  two 
principal  activities,  a review  of  the  simulator  training  research  lit- 
erature, and  a survey  of  representative  Air  Force  simulator  training 
activities.  The  literature  review  concentrated  upon  the  identification  of 
factors  influencing  simulator  training  effectiveness  and  the  identifica- 
tion of  methods  suitable  for  determining  simulator  training  effectiveness. 
Likewise,  the  survey  of  simulator  training  activities  concentrated  upon 
corresponding  areas  of  interest;  the  identification  of  factors  that  were 
influencing  the  effectiveness  of  Air  Force  simulator  training  activities; 
and  the  effectiveness  of  the  simulator  training  activities  investigated 
during  the  study  and  the  methods  employed  by  the  Air  Force  in  determining 
their  effectiveness.  The  literature  reviews  preceded  the  survey  activi- 
ties . 

Based  upon  information  developed  during  the  literature  review,  inter- 
view outlines^  were  prepared  for  use  during  the  survey  of  Air  Force  simu- 
lator training  activities.  The  interviews  were  conducted  on  site  at  a 
number  of  Air  Force  bases  where  simulator  training  activities  were  under 
way.  Interviews  at  these  bases  were  conducted  with  a wide  range  of  per- 
sonnel involved  in  simulator  training,  including  aircrew  members  who  re- 
ceive training  in  simulators,  instructors  who  conduct  that  training,  super- 
visors who  manage  both  simulator  and  aircraft  training,  and  otlier  personnel 
such  as  ISD  Team  members  who,  by  virtue  of  their  assignment,  are  cognizant 
of  simulator  training  activities  and  of  the  various  factors  that  might 
influence  the  effectiveness  of  such  activities  at  each  of  the  bases  in- 
cluded in  the  survey. 


These  outlines  are  contained  in  the  Appendix. 
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Table  1 identified  the  Air  Force  Bases  visited  during  this  research 
and  the  simulators  involved  in  the  training  activities  investigated  at 
each.  These  bases  and  simulators  were  selected  to  represent  Air  Force 
simulator  training  activities  beyond  the  Undergraduate  Pilot  Training  (UPT) 
level.  UPT  simulator  training  activities  were  not  investigated  during  the 
present  research  because  the  nature  of  UPT  simulator  training  is  expected 
to  undergo  significant  change  in  the  near  future  with  the  introduction  of 
new  simulators  currently  being  procured  for  that  program. 

TABLE  1.  SIMULATOR  TRAINING  PROGRAMS  SURVEYED 


Air  Force 
Commands 

Combat 

Crew  Training 

Continuation  Training 

Ai. ''craft 

Location 

Aircraft 

Location 

ADCOM 

F-106 

Tyndall  AFB 

F-106 

Castle  AFB 

MAC 

C-5 

Alt us  AFB 

C-5 

Travis  AFB 

SAC 

B-52 

Carswell  AFB 

B-52 

Castle  AFB 

FB-111 

Plattsburgh  AFB 

TAC 

F-4 

Luke  AFB 

F-4 

Eglin  AFB 

A- 7 

Davis-Monthan  AFB 

The  simulator  training  programs  included  in  the  survey  were  selected 
to  be  representative  of  Air  Force  aircraft  and  mission  types.  Both  Com- 
bat Crew  Training  (CCT)  and  Continuation  Training  (CT)  simulator  programs 
weie  surveyed  for  the  Aerospace  Defense  Command  (ADCOM) , the  Military  Air- 
lif''  Command  (MAC),  the  Strategic  Air  Command  (SAC),  and  the  Tactical  Air 
Command  (TAC).  For  each  command,  the  same  simulators/aircraft  were  sur- 
veyed with  respect  to  both  CCT  and  CT,  e.g.,  the  MAC  portion  of  the  sur- 
vey concentrated  upon  the  C-5  aircraft  for  both  CCT  and  CT  activities. 

In  the  cases  of  SAC  and  TAC,  the  survey  of  CCT  activities  included  an  addi- 
tional aircraft  not  included  in  the  surveys  of  CT  activities.  This  was 
done  because  of  the  large  differences  in  aircraft  configuration  and  crewing 
assignments,  compared  with  the  other  aircraft  surveyed,  and  the  additional 
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aircraft  represented  (i.e.,  the  two-place  FB-111  for  SAC  and  the  single- 
place A-7  for  TAG).  It  was  thought  that  including  these  additional  aircraft 
might  yield  data  of  Interest  to  the  purposes  of  this  project  which  would 
not  apply  to  the  other  aircraft  included  in  the  survey.  Thus,  the  survey 
examined  six  CCT  and  four  CT  programs  of  the  four  commands  located  at  nine 
Air  Force  bases.  Six  different  aircraft  were  simulated  by  the  devices 
examined  during  the  survey.  For  several  of  these  aircraft,  different  models 
of  the  same  aircraft  were  represented  by  these  simulators,  e.g.,  the  B-52D 
and  the  B-52G. 

In  addition  to  the  interviews  conducted  at  the  locations  identified 
in  Table  1,  the  simulators  themselves  were  examined,  and  Air  Force  docu- 
ments describing  the  devices  and  the  manner  of  their  use  were  reviewed. 
Particular  afention  was  directed  to  seeking  documented  evidence  of 
simulator  training  effectiveness  at  each  location.  Wliere  no  documented 
evidence  of  the  effectiveness  of  simulator  training  could  be  found,  per- 
sonnel responsible  for  the  conduct  of  such  training  were  questioned  ex- 
tensively in  an  attempt  to  determine  the  perceived  value,  quantitative 
or  otherwise,  of  such  training  and  the  bases  for  their  perceptions.  As 
was  stated  earlier,  no  attempts  were  made  to  conduct  experimental  studies 
to  determine  the  effectiveness  of  simulator  training  activities. 

ORGANIZATION  OF  THE  REPORT 

This  report  is  divided  into  four  principal  sections,  of  which  the 
present  Introductory  section  is  the  first.  In  Section  II,  methods  of  deter- 
mining simulator  training  effectiveness  are  described,  and  the  findings 
from  the  present  research  concerning  representative  Air  Force  practices 
in  determining  simulator  training  effectiveness  are  presented.  Section 
III  summarizes  Information  obtained  during  the  recent  research  concerning 
factors  that  influence  Air  Force  simulator  training  programs  and  summarizes 
the  results  of  the  survey  of  the  literature  related  to  those  factors.  Con- 
clusions and  recommendations  based  upon  the  information  developed  during 
this  research  are  to  be  found  in  Section  4. 
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•V  II.  METHODS  USED  TO  DETERMINE  SIMULATOR  TRAINING  EFFECTIVENESS 

t 
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INTRODUCTION 

The  increased  emphasis  within  the  Air  Force  during  the  past  few  years 
upon  tlie  use  of  simulators  in  aircrew  training  has  been  noted  in  Section  1. 
Because  of  the  greater  role  simulators  are  now  playing  in  Air  Force  pre- 
paredness, increased  attention  must  be  directed  to  assuring  that  simulators 
are  appropriately  designed  and  used, and  to  determining  the  extent  to  which 
they  are  effective  in  the  achievement  of  designated  training  goals.  The 
determination  of  simulator  training  effectiveness  often  has  been  neglected, 
however,  because  systematic,  controlled  studies  involving  these  devices 
sometimes  interfere  with  training  schedules  or  because  cognizant  personnel 
may  lack  the  time  and  resources  necessary  to  conduct  such  studies.  In 
addition,  most  personnel  responsible  for  simulator  use,  understandably,  are 
not  trained  as  behavioral  scientists, and  are  ill  prepared  with  respect  to 
knowledge  of  experimental  design  and  performance  measurement  methodologies 
to  validate  their  simulator  training  programs.  Consequently,  some  simu- 
lator users  are  forced  to  assume  that  their  simulator  training  programs 
are  effective,  an  assumption  that  may  or  may  not  be  correct. 

During  the  conduct  of  the  research  described  in  this  report,  infor- 
mation concerning  the  effectiveness  of  the  simulator  training  activities 
surveyed  was  sought.  Simulator  Instructors,  simulator  training  supervisors, 
and,  where  possible,  ISD  team  members  and  senior  Squadron  and  Wing  per- 
sonnel cognizant  of  simulator  training  activities  over  an  extended  time 
period  were  questioned  concerning  the  known  or  established  effectiveness 
of  the  simulator  training  being  conducted  at  each  location.  Specific 
inquiries  were  made  concerning  the  existence  of  documented  evidence  of 
any  kind  as  to  the  effectiveness  of  entire  simulator  training  programs 
and/or  any  element  of  those  programs.  Other  questions  concerned  the  man- 
ner in  which  the  worth  of  simulator  training  had  been  determined  and,  in 
those  cases  in  which  an  established  simulator  training  program  had  been 
modified,  what  procedures  wore  employed  to  provide  assurance  that  such 
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modification  had  a positive  rather  than  a negative  effect  upon  the  pro- 
gram's training  value. 

In  general,  it  was  found  that  most  Air  Force  simulator  training  pro- 
grams had  not  been  subjected  to  formal  studies  that  would  establish  their 
training  effectiveness  in  quantitative  terms.  The  evidence  of  simulator 
training  effectiveness  that  was  found  often  had  been  gathered  informally 
and  suggested  only  a general  value  for  simulator  training.  In  no  case 
was  documented  evidence  found  indicating  a demonstrated  benefit,  where 
particular  training  objectives  were  concerned,  of  training  in  a simulator 
versus  training  in  an  aircraft  (or  other  media).  A comment  made  by  one 
of  the  individuals  interviewed  illustrates  the  lack  of  emphasis  at  some 
bases  upon  determining  simulator  training  effectiveness:  "No  one  ever 

asked  me  before,  'How  do  you  know  the  simulator  training  is  effective?'" 
More  often,  however,  informal  evidence  at  least  suggestive  of  the  con- 
tribution of  simulator  training  to  overall  CCT  and  CT  program  effective- 
ness was  found.  The  manner  in  which  that  evidence  was  being  obtained  by 
Air  Force  personnel  is  discussed  below. 

TWO  APPROACHES  TO  INVESTIGATING  TRAINING  EFFECTIVENESS 

In  examining  the  methods  and  procedures  available  for  use  in  deter- 
mining the  effectiveness  of  simulator  training,  a distinction  must  be 
made  between  two  approaches  to  such  investigations:  (1)  studies  to  deter- 

mine the  effectiveness  of  a particular  simulator  training  activity:  and 
(2)  research  on  factors  Influencing  simulator  training  effectiveness. 

The  first  approach  is  concerned  with  validating  a particular  use  of  a 
particular  device  in  meeting  a particular  training  requirement.  Per- 
sonnel who  undertake  such  single  purpose  studies  typically  have  limited 
training  in  experimental  design  and  data  analysis  techniques,  and  the 
designs  they  employ  tend  to  be  unsophisticated  and  Involve  no  more  than 
two  groups  of  trainees,  usually  groups  of  opportunity.  Since  the  chief 
concern  in  the  conduct  of  these  studies  usually  is  practical  significance 
or  economic  gain  rather  than  statistical  significance,  formal  statistical 
tests  of  significance  often  are  not  employed,  and  study  results  are  judged 
simply  by  Inspection  of  the  data  or  by  practical  and  economic  considera- 
tions . 


10 


The  second  approach,  research  on  factors  influencing  simulator  training 
effectiveness,  almost  invariably  is  undertaken  by  personnel  trained  in  research 
methods.  These  studies  also  may  be  relatively  simple  in  design  and  analy- 
tic requirements.  Since  the  purpose  of  such  investigations  is  to  gain  in- 
formation which  will  be  used  to  design  simulators  and  simulator  training 
programs,  however,  more  sophisticated  experimental  designs  and  data  analysis 
techniques  involving  multiple  values  of  one  or  more  variables  or  factors 
under  study  are  more  likely  to  be  used  to  increase  experimental  efficiency. 
Because  of  the  complexity  of  such  designs  and  the  difficulty  of  interpreting 
both  single  factor  results  and  their  interactive  effects,  statistical  tests 
usually  are  necessary.  Sometimes,  though,  statistically  significant  differ- 
ences in  results  between  treatment  groups  that  are  important  to  understanding 
the  factors  involved  may  have  little  practical  or  economic  significance. 

Air  Force  personnel  responsible  for  ongoing  training  programs  are  con- 
cerned primarily  with  the  first  of  these  approaches,  that  of  determining 
the  effectiveness  of  a particular  simulator  training  activity.  Personnel 
interested  in  the  second,  broader  approach,  research  on  factors  which  in- 
fluence simulator  training  effectiveness,  are  more  likely  to  be  associated 
with  research  and  development  agencies  such  as  the  Air  Force  Human  Resources 
Laboratories.  The  present  survey  of  Air  Force  simulator  training  was 
directed  primarily  toward  the  activitit>s  of  operational  and  management  per- 
sonnel, rather  than  toward  trained  research  personnel  with  more  general 
Interests.  The  following  section  of  this  report  describes  techniques  em- 
ployed by  the  personnel  surveyed  to  determine  the  effectiveness  of  the 
particular  training  program  for  which  tliey  are  responsible.  Also  described 
are  other  techniques,  or  study  design  models,  that  could  be  considered  by 
personnel  responsible  for  future  studies  of  the  effectiveness  of  particular 
simulator  training  programs. 

.STUDY  DESIGN  MODELS 

A number  of  study  design  models  can  be  used  to  determine  simulator 
training  effectiveness,  some  of  which  are  more  suitable  to  the  task  than 
others.  .No  single  design  is  suitable  for  all  simulator  training  effective- 
ness studies,  however.  While  factors  such  as  the  availability  of  personnel 
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and  other  resources,  time  limitations,  competing  training  activities,  and 
administrative  constraints  sometimes  necessitate  use  of  a less  than  optimum 
study  design,  the  Investigator  must  guard  against  using  a design  that  cannot 
yield  information  suitable  to  his  needs. 

A number  of  study  designs  that  have  been  used  in  attempts  to  determine 
the  effectiveness  of  simulator  training  are  listed  below,  along  with  comments 
concerning  the  circumstances  surrounding  their  use  and  the  general  nature 
or  relevance  of  the  information  they  yield.  The  study  designs  are  described 
in  terms  of  simple  models.  Along  with  the  description  of  each  model  is  a 
summary  of  the  uses  made  of  it  by  the  Air  Force  in  determining  the  effective- 
ness of  the  simulator  training  program  surveyed  during  the  present  research. 
In  general,  the  models  are  presented  in  order  of  their  overall  merit  for  use 
in  determining  simulator  training  effectiveness. 

The  Transfer  of  Training  Model 

The  Transfer  of  Training  Model  generally  is  the  study  design  model  most 
appropriate  to  determine  whether  simulator  training  has  improved  subsequent 
operational  performance.  The  model  is  basfed  upon  the  basic  concept  under- 
lying the  use  of  simulators,  transfer  of  training.  Transfer  of  training  is 
a phenomenon  that  occurs  when  the  existence  of  a previously  learned  behavior 
or  skill  has  an  influence  upon  the  acquisition,  performance,  or  relearning 
of  a second  behavior  or  skill.  Thus,  if  a behavior  is  learned  in  an  air- 
craft simulator,  and  the  existence  of  that  behavior  or  the  fact  of  its  having 
been  learned  has  an  Influence  upon  the  subsequent  acquisition,  performance, 
or  relearning  of  behavior  in  an  aircraft,  transfer  is  said  to  have  occurred. 
The  influence  of  simulator  training  can  be  either  positive  or  negative.  In 
practice,  the  influence  may  be  positive  with  respect  to  some  behaviors  and 
negative  or  neutral  with  respect  to  others.  If  the  sum  of  these  influences 
is  positive,  use  of  the  simulator  can  reduce  dependence  upon  operational 
aircraft  during  training  by  facilitating  the  learning  of  tasks  that  must  be 
performed  in  those  aircraft. 

In  its  simplest  form,  the  Transfer  of  Training  Model  involves  two  groups 
of  trainees;  an  experimental  group  which  receives  simulator  training  prior 
to  further  training  or  performance  testing  in  the  aircraft;  and  a control 
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group  which  receives  nil  of  its  training  in  the  aircraft.  Alternatively, 
the  experimental  group  could  be  participants  in  a newly  developed  simulator 
training  program,  and  the  control  group  could  be  participants  in  an  existing 
simulator  program.  This  design  permits  differences  in  performance  in  the  air- 
craft between  the  experimental  and  control  groups  to  be  attributed  to  the 
influence  of  training  received  by  the  experimental  group.  The  groups  must 
be  equated,  of  course,  in  terms  of  relevant  prior  training  and  experience. 

In  all  research  designs  involving  control  groups,  consideration  must 
be  given  to  whether  the  control  "treatment"  itself  might  influence  that 
group's  subsequent  performance  in  the  criterion  situation.  The  influence 
could  be  facilitative , e.g.,  a period  of  rest  for  the  control  group  wliile 
the  experimental  group  engages  in  fatiguing  or  stressful  training;  or 
debilitating,  e.g.,  a period  of  fatiguing  or  stressful  activity  such  as 
operational  missions  or  extended  duty  required  only  of  the  control  group 
because  of  their  availability  for  additional  assignments.  Particular  care 
should  be  taken  that  members  of  neither  group  engage  in  flying  or  related 
operational  activities  likely  to  influence  their  performance  on  criterion 
tasks  and  thus  invalidate  experimental  and  control  group  comparisons. 

During  the  present  survey  of  the  Air  Force  simulator  training  activities, 
no  examples  were  found  of  the  use  of  the  Transfer  of  Training  Model  to  de- 
termine the  effectiveness  of  any  of  the  ten  simulator  training  programs 
surveyed . 

The  Self-Control  Transfer  Model 

Variations  of  the  basic  transfer  model  described  above  have  been  dis- 

1 2 3 

cussed  by  Gagn6,  Foster,  and  Crowley;  Woodworth  and  Schlosberg;  Murdock; 

and  Campbell  and  Stanley.^  One  variation  of  particular  interest  is  useful 

V.agne,  R.  M. , Foster,  Harriet,  and  Crowley,  Miriam  E.  The  Measurement 
of  Transfer  of  Training.  Psychological  Bulletin,  1948,  ^5,  97-130. 

2 

Woodworth,  R.  S.  and  Schlosberg,  H.  Experimental  Psychology.  New  York: 
Henry  Holt  and  Company,  1960. 

3 

Murdock,  B.  B.,  Jr.  Transfer  Designs  and  Formulas,  Psychological 
Bulletin,  1957,  313-326. 

4 

Campbell,  D.  T.  and  Stanley,  J.  C.  Experimental  and  Quasi-Experimental 
Designs  for  Research.  Chicago:  Rand  McNally,  1963- 

13 


i 


in  a situation  in  which  a device  might  be  used  at  an  intermediate  stage  of 
training,  i.e.,  when  operational  training  is  interrupted  for  a period  of 
training  in  a simulator.  In  such  a situation,  the  students  in  the  experi- 
mental group  could  serve  as  their  own  controls,  and  performance  data  obtained 
in  the  operational  aircraft  immediately  following  simulator  training  could 
be  compared  with  similar  data  obtained  in  the  aircraft  immediately  prior  to 
engaging  in  simulator  training.  The  difference  in  these  two  sets  of  per- 
formance data,  then,  could  be  attributed  to  the  intervening  simulator  train- 
ing program.  The  results  of  such  a study  might  be  suspect,  however,  because 
of  the  confounding  effects  of  forgetting  (or  reminisi-ence) , particularly  if 
there  was  a significant  time  interval  between  initial  and  subsequent  prac- 
tice in  the  operational  vehicle. 

Although  a number  of  opportunities  exist  to  employ  the  Self-Control 
Transfer  Model  to  determine  whether  various  Air  Force  simulator  training  ] 

programs  are  effective,  no  instances  were  found  during  the  present  survey  I 

of  Air  Force  training  in  which  this  model  has  been  employed  in  any  formal  ; 

sense.  Two  examples  of  informal  use  of  the  model  by  Air  Force  personnel  i 

are  described  below.  However,  the  nature  of  the  data  employed  in  each  case, 
i.e.,  the  absence  of  negative  feedback  in  one  case  and  unsubstantiated 

opinion  in  the  other,  made  questionable  in  these  Instances  the  value  of  the  j 

model  and  the  interpretations  of  the  results.  j 

Flight  performance  of  aircrews  coming  from  other  bases  to  participate  j 

in  one  simulator  continuation  training  program  is  evaluated  in  flight  by  ; 

examiners  at  their  home  station  from  time  to  time.  Presumably,  some  of  these 

evaluations  take  place  before  and  some  after  participating  in  continuation  j 

training  and  could  provide  evidence  of  the  effectiveness  of  such  training.  ! 

There  is  no  system  for  feedback  of  information  from  the  home  station  con-  fi 

cerning  the  performance  of  the  returning  aircrews  that  could  be  attributed 
to  the  training  they  received.  However,  the  absence  of  negative  feedback 
has  been  cited  as  evidence  that  such  training  is  effective. 

The  simulator  used  for  continuation  training  at  another  base  became 
available  only  recently,  so  it  now  constitutes  an  additional  capability  that 
might  be  used  to  enhance  the  operational  readiness  of  the  assigned  pilots. 

No  formal  attempt  has  been  made  to  determine  whether  the  pilots  now  receiving 
the  additional  training  are  more  proficient  than  they  were  before  the  device 
became  available,  however.  A senior  Instructor  pilot  expressed  the  strong 
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opinion  that  there  had  been  specific  improvements  in  squadron  pilots'  in- 
flight capabilities  following  introduction  of  simulator  training,  particular! 
with  respect  to  intercept  missions,  but  no  data  had  been  assembled  to  verify 
this  opinion. 


The  Pre-Existing  Control  Transfer  Model 

There  are  instances  in  which  a concurrently  trained  control  group  may 
not  be  necessary.  For  example,  when  simulator  training  is  added  to  an 
existing  training  program,  or  when  a new  simulator  training  program  replaces 
an  old  one,  student  performance  data  from  the  existing  or  older  program 
can  be  compared  with  comparable  data  from  the  new  program  to  determine 
the  latter's  effectiveness.  For  such  a comparison  to  be  valid,  the  pre- 
existing data  must  have  been  gathered  under  conditions  which  would  have 
been  applicable  to  a control  group  trained  concurrently  with  the  experi- 
mental group.  A disadvantage  of  the  Pre-Existing  Control  Transfer  Model 
is  that  differences  in  performance  between  the  two  groups  may  be  the  result 
of  chances  which  have  occurred  in  the  population  during  the  time  between 
which  the  experimental  and  control  students  were  drawn. 

Historic  records  at  a number  of  the  bases  Involved  in  the  present 
survey  indicate  that  the  hours  programmed  for  flight  training  were  re- 
duced following  introduction  of  an  additional  simulator  capability  (e.g. , 
a visual  display  attachment)  or  following  an  ISD  activity  related  to 
increased  use  of  simulation.  Wliile  it  is  not  possible  to  attribute  those 
reductions  in  scheduled  flying  time  solely  to  increased  effectiveness  of 
the  simulator  training,  it  is  at  least  likely  that  such  reductions  would 
not  have  been  acceptable  to  the  command  involved  had  not  the  reduction 
presumably  been  offset  by  increased  simulator  training  effectiveness. 
Unfortunately,  hard  data  generally  do  not  exist  that  can  be  used  to  demon- 
strate that  these  reductions  were  Justified  by  a more  effective  simulator 
training.  Therefore,  it  is  possible  that  at  least  a portion  of  the  reduc- 
tion in  flying  hours  which  accompanied  changes  in  some  simulator  training 
programs  may  have  been  based  upon  administrative  action  rather  than  upon 
actual  simulator  training  benefits. 

When  training  in  a cockpit  procedures  trainer  was  added  to  one  of  the 


Combat  Crew  Training  programs  surveyed,  the  amount  of  time  devoted  to  pro- 
cedural training  in  the  simulator  was  reduced.  The  basis  for  the  reduc- 
tion was  the  informal  observation  that  the  simulator  was  no  longer  required 
for  the  development  of  certain  skills  associated  with  the  location  and 
activation  of  selected  cockpit  controls.  This  reduction  in  the  need  for 
the  simulator  to  be  used  for  selected  training  previously  conducted  in  it 
permitted  additional  tasks  to  be  Included  in  the  simulator  training  pro- 
gram within  the  same  scheduled  time,  thus  increasing  the  scope  of  simu- 
lator training  in  that  particular  program. 

Personnel  at  another  base  reported  instances  in  which  CCT  students 
had  not  participated  in  the  normally  prescribed  simulator  training  before 
their  first  flight  in  the  aircraft  (reasons  not  given).  The  reported 
result  was  that  the  first  aircraft  flight  was  considered  only  a familiari- 
zation flight  during  which  the  training  usually  conducted  on  such  flights 
could  not  be  accomplished.  Although  no  formal  attempt  was  made  to  assess 
the  relative  skill  of  these  students  vis-a-vis  students  who  received  the 
prescribed  simulator  training,  the  instructors  involved  noted  a significant 
difference  in  their  knowledge  and  ability  in  the  air.  Whether  their  fail- 
ure to  receive  the  prescribed  early  simulator  training  led  to  a detectable 
difference  in  performance  at  the  end  of  the  training  program  for  the  few 
students  Involved  is  not  known,  nor  is  it  known  whether  the  loss  of  initial 
simulator  training  had  to  be  compensated  by  additional  training  time  in 
the  aircraft. 

The  Uncontrolled  Transfer  Model 

There  are  circumstances  in  which  a separate  control  group  cannot  be 
employed,  the  Self-Control  or  the  Pre-Existing  Control  Transfer  Models  are 
Inappropriate,  and  suitable  control  data  do  not  exist.  Such  circumstances 
might  be  dictated  by  political,  administrative  or  safety  considerations. 

For  example,  it  might  be  unacceptable  to  "penalize"  members  of  one  group 
by  requiring  that  they  undergo  a different  and  possibly  inferior  train- 
ing program.  In  some  Instances,  a control  group  simply  may  not  be  feasible, 
e.g. , the  effectiveness  of  lunar  landing  simulators  could  not  be  determined 
by  employing  a no-simulator  training  control  group  of  astronauts. 
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Wlien  a control  group  cannot  be  employed  and  suitable  control  data  do 
not  exist,  simulator  training  effectiveness  can  be  established  by  deter- 
mining whether  students  can  perform  a particular  task  in  the  operational 
vehicle,  following  its  learning  in  the  simulator,  without  an  opportunity 
to  learn  that  task  in  the  operational  vehicle.  Data  gathered  under  this 
model  will  be  suspect,  since  it  cannot  be  shown  conclusively  that  the 
behaviors  involved  can  be  attributed  solely  to  simulator  training. 

Nevertheless,  such  data  can  carrv  considerable  weight,  particularly  when 
a task  critical  to  flight  safety  is  involved  and  a plausible  case  can  be 
made  that  the  underlying  skills  probably  are  attributable,  at  least  in 
part,  to  the  simulator  training  program. 

Many  instances  were  found  among  the  Air  Force  programs  surveyed  in 
which  the  Uncontrolled  Transfer  Model  has  been  applied  to  determine  the 
effectiveness  of  a particular  simulator  training  program.  These  instances 
are  typified  by  simulator  training  followed  by  further  training  in  the 
aircraft,  during  which  trainee  performance  is  assessed  in  some  fashion. 

Although  almost  all  Air  Force  CCT  and  CT  programs  follow  some  variation 
of  this  sequence,  the  idea  of  using  the  resulting  aircraft  performance 
assessment  data  to  validate  simulator  training  effectiveness  is  not  pur- 
sued. In  fact,  the  in-flight  performance  assessment  is  more  likely  to 
reflect  a number  of  factors  other  than  simulator  training  effectiveness 
because  of  the  structure  of  the  overall  training  program  and  the  point 
during  flight  training  at  which  the  assessments  are  made. 

An  example  of  the  informal  use  by  the  Air  Force  of  the  Uncontrolled 
Transfer  Model  noted  during  the  present  survey  involved  the  A-7D  aircraft 
in  the  CCT  program  at  Davis-Monthan  Air  Force  Base.  This  program  can 
be  considered  to  be  an  application  of  the  present  Model  in  that  the  first 
flight  in  the  aircraft  can  bo  viewed  as  an  informal  examination  flight  in 
which  specific  skills,  those  associated  with  taking  off  and  landing  the 
aircraft,  can  bo  assessed. 

The  A-7D  is  a single  place  aircraft,  and  all  training  flights  in  it 
must  be  solo.  The  fact  that  the  first  flight  in  this  aircraft  in  the  CCT 
program  is  almost  invariably  successful  to  one  degree  or  another  suggests 
that  the  training  the  students  receive  prior  to  that  flight  is  effective. 

The  student  pilot  learns  to  take  off  and  land  the  aircraft  in  the  simulator 

i 
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using  the  Heads  Up  Display  (there  is  no  extra-cockpit  visual  display  on  the 
device).  On  his  first  flight  in  the  aircraft,  he  apparently  relies  upon 
I-  this  training  (and  his  previous  UPT  experience  in  the  T-38)  in  order  to 

take  off  and  land  the  A-7D  aircraft  without  a qualified  pilot  on  board. 

The  extent  to  which  the  simulator  training  contributes  to  his  ability  to 
perform  safely  during  this  first  flight  cannot  be  determined  more  pre- 
cisely using  the  Uncontrolled  Transfer  Model,  however. 

A number  of  other  assumptions  of  the  value  of  simulator  training  were 
cited  during  the  survey  which  could  be  considered  to  be  based  upon  appli- 
cation of  the  Uncontrolled  Transfer  Model.  Each  involved  a particular 
task  for  which  training  was  conducted  in  the  simulator  and  for  which 
benefits  of  that  training  were  presumably  noted  in  the  aircraft  during 
subsequent  training  flights.  For  example,  performing  an  engine  start 
for  the  first  time  in  a simulator  is  a time  consuming  activity.  However, 
pilots  who  have  mastered  that  task  in  the  simulator  typically  perform 
their  initial  engine  start  in  the  aircraft  fairly  rapidly. 

The  Simulator-to-Slmulator  Transfer  Model 

Many  studies  of  the  Influence  of  specific  features  of  simulators  or 
simulator  training  programs  upon  their  training  effectiveness  Involve 
transfer  of  training  from  one  simulator  to  another  rather  than  to  opera- 
tional equipment.  For  example,  a study  of  the  role  of  platform  motion 
upon  pilot  training  effectiveness  might  Involve  training  in  a simulator 
without  motion,  followed  by  performance  evaluation  in  the  same  or  another 
simulator  with  motion.  This  design,  which  might  be  employed  when  no  | 

aircraft  is  available,  is  based  upon  an  assumption  of  equivalence,  so  far 
as  criterion  performance  is  concerned,  between  the  second  simulator  and 
the  unavailable  aircraft.  This  is  a tenuous  assumption,  and  it  is  con- 
ceivable that  conclusions  based  upon  transfer  of  training  data  derived 
through  use  of  this  model  could  be  erroneous.  In  any  event,  no  instances 
were  found  during  the  present  survey  in  which  the  Slmulator-to-Slmulator 
Transfer  Model  had  been  employed  to  determine  the  effectiveness  of  an  Air 
Force  simulator  training  program. 

There  is  one  situation  in  which  the  Slmulator-to-Slmulator  Transfer 
Model  is  appropriate.  That  situation  exists  when  the  second  simulator 
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is  the  criterion  vehicle.  For  example,  the  effectiveness  of  training  in 
a part-task  training  device  can  be  determined  by  measurement  of  perform- 
ance in  a full-mission  simulator  if  the  objective  of  such  part-task  train- 
ing is  to  reduce  the  use  of  the  more  complex  device.  In  this  situation, 
it  would  be  presumed  that  performance  in  the  simulator  would  involve  inter- 
mediate training  objectives,  with  the  final  objectives  relating  to  sub- 
sequent performance  in  an  operational  vehicle.  The  situation  described 
here  is  an  application  of  one  of  the  Transfer  Models  described  earlier, 
with  the  second  simulator  equating  to  the  operational  vehicle  itself. 

The  Backward  Transfer  Model 

Another  simulator  evaluation  design,  known  as  backward  or  inverse 
transfer  of  training,  is  based  upon  the  transfer  of  training  concept  and 
has  been  described  by  Adams  and  McAbee.^  In  a backward  transfer  study,  an 
operator  who  already  has  demonstrated  mastery  of  relevant  training  objec- 
tives in  the  operational  vehicle  Is  "transferred"  to  the  simulator,  where 
he  is  required  to  perform  tasks  corresponding  to  those  he  has  mastered 
operationally.  If  he  can  perform  such  tasks  to  criterion  levels  without 
practl-e  in  the  simulator,  backward  transfer  is  said  to  have  occurred, 
and  this  fact  is  taken  as  evidence  that  transfer  in  the  simulator -to- 
vehicle  sequence,  although  of  unknown  quantity,  will  be  positive. 

The  backward  transfer  design  should  be  used  with  caution  for  at  least 
three  reasons:  (1)  positive  results  assume  (possibly  incorrectly)  that  a 

suitable  training  program  exists  for  the  simulator;  (2)  experienced  per- 
sonnel already  proficient  at  operational  tasks  may  have  highly  generalized 
skills  not  possessed  by  recent  training  program  graduates  and  may  be  able 
to  transfer  to  the  device  because  of  such  general  skills  rather  than  skills 
needed  to  operate  a particular  vehicle  or  perform  a particular  mission; 
and  (3)  the  simulator  may  he  suitable  designed  for  the  evocation  of  a 
particular  set  of  behaviors  by  skills  performers,  but  may  lack  the  cues 
essential  to  the  development  of  those  behaviors. 


Adams,  J.  A.  and  McAbee,  W.  H.  A Program  for  a Functional  EvaUiation 
of  the  GAM-83  Melpar  Trainer.  Report  No.  APGC-TN-hl -41 , USAF  Air  Proving 
Grounds,  Eglin  AFB,  FL,  October  19bl. 
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While  bnrkwarci  transfer  data  should  not  be  the  sole  justification  for 
adopting  a particular  simulator,  one  would  be  hesitant  to  recommend  a pro 
gram  involving  a device  which  could  not  bo  handled  by  competent  pilots. 
Negative  results  could  bo  misleading  also,  however.  It  is  possible  for 
some  t.isks  to  bo  perforraeil  in  the  aircraft  bv  personnel  who  use  cues  not 
present  in  the  simulator,  and  therefore,  they  could  bo  unable  to  perform 
such  tasks  in  the  simulitor  without  training  in  it,  while  the  same  simu- 
lator mav  provide  other  cues  which  trainees  can  learn  to  use  to  perform 
those  same  tasks  in  the  simulator  for  subsequent  transfer  to  the  aircraft 

No  evidence  was  fiMind  of  the  application  of  this  model  in  order  to 
assess  simulator  training  effectiveness  in  the  Air  Force  programs  sur- 
veyed. In  several  simulator  training  programs,  there  were  anecdotal 
reports  of  pilots  having  difficulty  flying  the  simulator  after  learning 
to  fly  the  aircraft,  but  there  was  no  evidence  that  such  information  had 
led  to  an  investigation  to  determine  whether  negative  transfer  might  be 
a significant  problem  in  these  programs. 

The  S tmiilator  Performance  Improvement  Model 

A presumably  essential  feature  of  an  effective  simulator  training 
program  is  Improvement  in  the  performance  of  trainees  in  the  simulator 
as  a result  of  training  they  receive  in  the  simulator.  If  such  improve- 
ment does  not  occur,  there  would  be  little  expectation  that  subsequent 
operational  performance  would  be  improved  as  a result  of  simulator  train- 
ing. Because  of  this  dependency  relationship,  improvement  in  performance 
in  the  simulator  often  is  cited  as  evidence  that  simulator  training  is 
effective.  This  typically  is  done  when  circumstances  preclude  the  em- 
ployment of  a transfer  model  to  determine  simulator  training  effective- 
ness. Examples  of  the  application  of  the  Simulator  Performance  Improve- 
ment Model  are  relatively  commiin,  e.g.,  the  evaluation  of  spacecraft 
simulator  training  programs  beiore  latinching  tmanned  spacecra  f t , and 
simulator  motion  system  training  effectiveness  studli's  conducted  in  the 
absence  of  an  in-flight  performance  evaluation  condition. 

Clearly,  there  are  circumstances  in  which  the  Simulator  Performance 
Improvement  Model  can  provide  the  best  av.iilable  e.stim.ite  of  whether  a 


r ^ 

I 


simulator  training  program  is  effective.  It  must  be  noted,  however,  that 
>,  this  model  yields  only  indirect  evidence  of  simulator  effectiveness.  It 

can  show  that  a necessary  condition  has  been  met,  but  it  does  not  justify 
the  conclusion  that  the  improved  performance  in  the  simulator  will  result 
in  improved  operational  performance.  This  model,  thus,  is  most  useful  in 
a negative  way:  if  no  improvement  occurs  in  the  simulator,  none  should  be 

expected  operationally. 

The  effectiveness  of  all  Air  Force  simulator  training  programs  included 
in  the  present  survey  was  judged,  at  least  implicitly,  according  to  the 
Simulator  Performance  Improvement  Model,  and  in  some  cases,  such  judgments 
were  the  chief  basis  for  determining  a program's  effectiveness.  At  one 
base,  for  example,  trainees  were  reported  to  be  more  proficient  in  the 
simulator  on  the  final  day  of  Continuation  Training  than  on  the  first,  al- 
though no  data  have  been  gathered  to  support  this  report.  The  effectiveness 
of  the  training  each  student  receives  is  considered  to  be  evidenced  by  the 
increased  simulator  proficiency  he  exhibits  at  the  end  of  the  five-day  CT 
program.  At  another  base  where  Combat  Crew  Training  is  conducted,  the  chief 
basis  for  judging  simulator  effectiveness  is  the  changes  which  are  reportedly 
observed  to  occur  in  trainee  performance  during  such  training,  i.e.,  pilot 
trainees  become  proficient  at  flying  the  simulator. 

At  other  bases,  more  formal  attempts  are  made  to  verify  that  trainee 
performance  improves  during  the  course  of  simulator  training.  For  example, 
in  one  of  the  CCT  programs  surveyed,  personnel  from  the  ISD  team  sometimes 
fly  with  a student  in  the  simulator  to  "verify"  that  he  has  met  the  training 
objectives  specified  for  the  preceding  simulator  training  flights.  Person- 
nel from  the  Stan/Eval  Group  also  give  checkrldes  in  the  simulator  at  fixed 
points  during  training,  and  proceeding  to  subsequent  training  activities  is 
dependent  upon  passing  such  checkrldes.  Since  the  students  involved  have 
been  engaged  in  a variety  of  training  activities,  including  training  in  the 
aircraft  in  some  cases,  performance  during  such  a checkride  in  the  simulator 
cannot  always  be  attributed  solely  to  earlier  training  received  in  the 
simulator,  however. 

The  Simulator  Performance  Improvement  Model  was  cited  by  personnel 
who  developed  one  of  the  CCT  programs  included  in  the  survey  as  the  basis 
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for  determining  how  much  simulator  training  to  Include  in  the  overall 
training  program.  They  reported  that  their  simulator  training  was  validated 
through  an  examination  of  trainee  grade  slips  in  the  simulator.  When 
grade  slips  indicated  that  performance  in  the  simulator  no  longer  could 
be  improved  with  more  simulator  training,  the  amount  of  training  in  the 
simulator  the  program  then  contained  was  considered  appropriate,  and 
the  use  being  made  of  the  simulator  was  judged  to  be  effective  on  that 
basis . 

The  Simulator  Fidelity  Model 

When  data  describing  trainee  performance  are  not  readily  available, 
other  kinds  of  data  thought  to  reflect  simulator  training  effectiveness 
may  be  sought.  Several  models  have  been  employed  under  such  circumstances 
to  generate  data  related  to  the  simulator  Itself  or  to  the  manner  of  its 
use.  One  of  these,  the  Simulator  Fidelity  Model,  yields  data  which  des- 
cribe the  simulator  in  terms  of  its  fidelity,  l.e.,  the  physical  corres- 
pondence between  it  and  the  operational  vehicle,  equipment  or  facility. 

Use  of  this  model  is  based  upon  the  assumption  that  a high  fidelity  simu- 
lator will  yield  high  transfer;  a low  fidelity  simulator  will  yield  less — 
or  even  negative — transfer. 

The  Simulator  Fidelity  Model  often  has  been  used  as  an  expedient  when 
data  reflecting  trainee  performance  could  have  been  obtained,  although  the 
model  also  has  been  used  when  no  other  kinds  of  data  were  available.  The 
model  has  wide  appeal  among  operational  personnel  who  are  not  familiar  with 
the  complexities  of  transfer  of  training,  and  who  lack  training  in  experi- 
mental methodology  and  performance  measurement.  It  can  be  employed  by 
anyone  familiar  with  the  operational  vehicle,  does  not  require  test  sub- 
jects and  other  resources,  and  is  based  upon  popularly  accepted  theoretical 
constructs  underlying  transfer  of  training,  e.g.,  Osgood's^  transfer  surface 
illustrating  the  assumed  relationship  between  stimulus  similarity, 
response  similarity,  and  transfer.  Systematic,  analytic  procedures  have 

^Osgood,  C.  E.  Method  and  Theory  in  Experimental  Psychology.  New  York: 
Oxford  University  Press,  1951. 


been  developed  for  the  employment  of  this  model  that  take  into  account 
fidelity  of  both  the  stimuli  the  simulator  presents  to  the  trainee  and 
the  responses  he  makes  to  those  stimuli.^  While  such  procedures  Increase 
the  objectivity  of  the  simulator  fidelity  data  yielded  by  the  model,  they 
do  not  overcome  the  basic  deficiency  of  the  model  itself;  it  yields  a 
measure  that  may  be  unrelated  to  operational  trainee  performance. 

Data  describing  simulator  fidelity  might  be  used  as  a partial  basis  for 

predicting  simulator  training  effectiveness,  but  its  use  for  determining 

2 

simulator  training  effectiveness  is  inappropriate.  Bryan  and  Regan  have 
noted  that  a simulator  can  be  a very  faithful  copy  of  operational  equip- 
ment and  be  either  effective  or  ineffective  with  respect  to  a particular 
training  requirement.  In  fact,  well  designed  training  equipment  may  deviate 

Intentionally  from  fidelity  in  order  to  promote  learning.  In  an  even  stronger 
3 

criticism,  Adams  stated  that  equating  training  effectiveness  with  fidelity 
is  a cover-up  for  our  ignorance  about  transfer  and  leads  to  the  development 
of  possibly  unnecessarily  costly  devices.  In  any  event,  the  Simulator  Fidel- 
ity Model  ignores  the  manner  in  which  a device  is  used  and  the  objectives 
of  device  training.  These  two  considerations  underlie  any  determination 
of  simulator  training  effectiveness. 

In  spite  of  the  significant  shortcomings  of  the  Simulator  Fidelity  Model 
as  a design  for  determining  simulator  training  effectiveness,  it  was  widely 
used  for  that  purpose  at  the  Air  Force  bases  visited  during  the  present  survey. 

A tendency  was  noted  during  the  survey  for  many  Air  Force  personnel  to  equate 
physical  correspondence  between  the  device  and  the  aircraft  with  training 
effectiveness  and  to  judge  effectiveness  purely  on  the  basis  of  that  corres- 
pondence. The  judged  fidelity  of  several  of  the  simulators,  particularly 
those  that  included  motion  simulation,  was  cited  as  evidence  of  the  effective- 
ness of  the  simulator  training  available  using  those  devices.  Further,  the 

'^Caro,  P.  W.  Equipment-Device  Task  Commonality  Analysis  and  Transfer 
of  Training.  Technical  Report  70-7,  Human  Resources  Research  Organization, 
Alexandria,  VA,  .Tune  1970. 

2 

Bryan,  G.  L.  and  Regan,  ,1.  J.  Training  System  Design.  In  Van  Cott,  H.  P. 
and  Kinkade,  R.  G.  (Ed.),  Human  Engineering  Guide  to  Equipment  Design  (Rev.  ed.). 
Washington:  Government  Printing  Office,  1972. 
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Adams,  J.  A.  Research  and  the  Future  of  Engineering  Psychology.  American 
Psychologist . 1972,  y,  615-h22. 
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more  complete  simulation  expected  to  be  obtained  through  future  modifica- 
tion of  some  of  the  devices,  e.g.,  the  addition  of  radar  land  mass  simu- 
lation to  the  FB-lll  simulator,  was  cited  as  evidence  that  future  simula- 
tor training  would  be  even  more  effective. 

The  emphasis  upon  simulator  fidelity  as  a basis  for  judging  simulator 
training  effectiveness  is  illustrated  by  the  requirement  of  some  Air  Force 
regulations  that  simulator  training  programs  be  validated  by  determining 
whether  the  simulators  used  in  those  programs  are  being  maintained  in  a 
manner  that  permits  faithful  simulation  of  the  aircraft  Involved.  For 
example,  SACR  50-46  specified  that,  quarterly,  standardization  personnel 
must  evaluate  each  SAC  simulator  to  identify  operational  serviceability 
problems.  The  assumption  is  made  that  a simulator  that  passes  this  evalu- 
ation, and  which  is  used  in  accordance  with  specified  procedures,  is  ef- 
fective. It  is  understood  that  emphasis  during  these  evaluations  is  upon 
verifying  that  the  simulator  operates  within  its  design  limitations  with 
respect  to  fidelity  of  simulation. 

The  Simulator  Training  Program  Analysis  Model 

Another  model,  sometimes  employed  when  trainee  performance  data  are 
not  readily  obtainable,  is  the  Simulator  Training  Program  Analysis  Model. 
Use  of  this  model  Involves  analysis  of  the  way  the  simulator  is  used  to 
determine  whether  the  training  program  is  well  designed,  is  directed  toward 
the  attainment  of  appropriate  training  objectives,  and/or  employs  modern  or 
innovative  training  techniques.  While  use  of  this  model  can  pinpoint 
possible  factors  limiting  the  effectiveness  of  simulator  training  in  a par- 
ticular Instance,  it  will  not  indicate  whether  such  training  is  effective. 
Use  of  this  model  in  conjunction  with  the  previously  discussed  Simulator 
Fidelity  Model  can  be  particularly  helpful  In  optimizing  simulator  training 
effectiveness,  but  determining  the  extent  of  that  effectiveness  must  be 
accomplished  through  use  of  other  models. 

Jeantheau^  suggests  using  these  two  analytic  models  In  combination  to 

^Jeantheau,  G.  G.  Handbook  for  Training  Systems  Eyalj^at^n . Tech- 
nical Report  NAVTRADEVCEN  66-C-0113-2,  Naval  Training  Device  Center,  Or- 
lando, FL,  January  1971. 
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obtain  a "qualitative"  assessment  of  simulator  training  effectiveness.  Such 
an  assessment  does  not  involve  measurement  of  any  kind  and  is  based  upon 
judgments  made  against  a priori  criteria  related  to  training  equipment  and 
processes  rather  than  to  trainee  performance.  Their  use  in  combination 
probably  would  not  yield  a more  valid  assessment  of  simulator  training 
effectiveness  tlian  would  either  used  alone. 

The  previously  cited  SAC  Regulation  59-46  specified  that  the  quarterly 
examination  of  SAC  simulators  by  standardization  personnel  will  include 
examination  of  the  simulator  training  program  "to  assure  that  only  current 
procedures  are  included  in  the  curriculum."  These  quarterly  examinations 
are  intended  to  verify  that  simulator  training  is  effective,  although  the 
regulation  does  not  contain  a requirement  for  the  standardization  personnel 
conducting  the  examination  to  inspect  trainee  grade  slips  or  other  evidence 
of  changed  trainee  performance  in  the  simulator  or  in  the  aircraft. 

Other  evidence  of  the  use  of  this  model  to  establish  the  effectiveness 
of  Air  Force  simulator  training  also  was  cited  by  several  of  the  personnel 
interviewed.  At  one  base,  changes  in  the  simulator  syllabus  to  include 
more  training  objectives  was  cited  as  evidence  that  simulator  training 
effectiveness  had  been  increased.  (In  that  program,  increased  effective- 
ness had  been  equated  with  increased  scope  of  the  training  offered.)  At 
another  base,  analyses  of  simulator  training  programs  by  ISD  and  other 
specially  trained  personnel  were  conducted  to  verify  that  simulator  train- 
ing was  effective,  i.e.,  that  the  content  of  the  training  program  corres- 
ponded with  the  recommendations  of  the  ISD  team. 

The  Opinion  Survey  Model 

There  are  circumstances  under  which  one  might  wish  to  determine  the 
effectiveness  of  simulator  training  when  operational  training  or  perfor- 
mance testing  is  not  feasible,  data  on  performance  in  the  simulator  are 
not  available,  and  the  simulator  and  its  training  program  cannot  be  ana- 
lyzed. For  example,  it  is  sometimes  necessary  to  make  decisions  concern- 
ing the  training  effectiveness  of  a newly  developed  simulator  (or  a simu- 
lator under  development)  before  a stuay  based  upon  another  design  model  can 
be  conducted. 
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Some  analyses  have  attempted  to  evaluate  simulators  by  asking  opera- 
tors, Instructors,  training  specialists,  and  even  students,  their  opinions 
concerning  simulator  effectiveness,  i.e.,  the  perceived  training  value  of 
the  device  or  certain  of  its  features,  or  the  probable  impact  upon  subse- 
quent operational  performance  of  training  in  the  simulator.  Such  opinion 
data  often  have  little  merit  and  even  when  of  value  may  easily  lead  to 
erroneous  conclusions.  The  writer  has  observed  instances  in  non-Air  Force 
training  programs  in  which  evaluations  based  upon  operators'  and  instruc- 
tors' opinions  yielded  effectiveness  estimates  unrelated  to  data  subse- 
quently obtained  in  transfer  studies  involving  the  simulators  in  question. 
Furthermore,  the  evaluations  were  often  expressed  without  regard  to  the  man- 
ner of  use  or  the  objectives  of  simulator  training.  Meister,  Sullivan, 
Thompson,  and  Finley^  found  that  estimates  of  simulator  training  effective- 
ness based  upon  instructors'  opinions  varied  as  a function  of  the  different 
instructors  expressing  the  opinions.  The  unreliability,  and  even  invalidity, 
of  determinations  of  simulator  training  effectiveness  based  upon  opinions 

of  instructors  and  other  experts  probably  are  due  in  part  to  attitude  fac- 

2 

tors  such  as  those  discussed  by  Mackie,  Kelley,  Moe , and  Mecherlkoff,  as 
well  as  to  the  inherent  unreliability  of  such  judgments.  In  the  final 
analysis,  simulator  training  effectiveness  must  be  established  by  trainee 
performance,  not  instructor,  operator,  or  trainee  opinions  about  the  device 
and  its  probable  usefulness. 

The  Opinion  Survey  Model  clearly  was  a commonly  used  method  for  deter- 
mining the  effectiveness  of  many  of  the  Air  Force  simulator  training  pro- 
grams surveyed.  Opinions  that  existing  simulator  training  programs  were 
effective  were  so  strong  at  several  of  the  bases  surveyed  that  no  need  for 
further  validation  of  those  programs  was  acknowledged.  At  one  of  these 
bases,  an  ISD  effort  was  underway  which  presumably  would  result  in  changes 
in  the  existing  simulator  training,  but  no  plans  to  validate  future  simulator 


Meister,  D.,  Sullivan,  D.  .1.,  Thompson,  E.  A.  and  Finley,  U.  L. 
Training  Effectiveness  Evaluation  of  Naval  Training  Devices.  Part  II:  A 
Study  of  Device  2F55A  (S-2E  Trainer)  Effectiveness.  Technical  Report 
NAVTRADeVcEN  66-C-0113-2,  Orlando,  FI.,  January  1971. 

2 

Mackie,  R.  R. , Kelley,  G.  R.,  Moe,  G.  L.  and  Mecherlkoff,  M.  Factors 
Leading  to  the  Acceptance  or  Rejection  of  Training  Devices.  Technical 
Report  NAVTRAEQUIPCEN  70-C-0276-1,  Naval  Training  Equipment  Center,  Orlando, 
FL,  August  1972. 
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training  were  included  in  the  ISD  program  except  to  have  the  program  in- 
spected by  the  ISD  team  itself  to  verify  that  it  was  being  conducted  as 
the  team  directed,  and  to  ask  trainees  for  their  opinions  regarding  its 
value  to  them. 

The  validation  of  the  effectiveness  of  one  CCT  program  surveyed  rests 
primarily  upon  feedback  from  students  obtained  from  end-of-training  cri- 
tiques. All  suggestions  contained  in  such  critiques  for  improving  the 
program  are  considered,  and  if  there  are  several  suggestions  along  the  same 
line,  change  in  the  simulator  training  program  becomes  probable.  Once  such 
changes  have  been  made,  validation  of  the  effectiveness  of  the  resultant 
simulator  training  would  be  provided  by  subsequent  reviews  of  student  cri- 
tiques. Presumably,  a program  that  receives  few  suggestions  for  change 
would  be  judged  to  be  effective,  and  further  improvements  would  not  be 
sought.  Such  a procedure  is  questionable. 

Validation  of  the  ef fectlv.gness  of  several  other  simulator  training 
programs  surveyed  was  reportedly  based  upon  syllabus  review  by  instructors 
and  critiques  by  trainees.  Suggested  changes  by  Instructors  almost  always 
lead  to  changes  in  such  programs,  so  there  are  frequent  changes  that  are 
not  subjected  to  external  validation.  All  changes  in  the  simulator  train- 
ing that  are  intended  to  increase  effectiveness  are  validated  by  obtaining 
a consensus  of  the  instructors  who  administer  it  to  students. 

Many  of  the  applications  of  the  Opinion  Survey  Model  noted  during  the 
present  survey  were  quite  Informal.  By  contrast,  a more  formal  evaluation 
of  the  limited  visual  system  (LVS)  for  the  C-5  and  C-141  flight  simulators 
at  Altus  AFB  has  been  conducted.^  Opinions  concerning  the  worth  of  the 
LVS  vis-a-vis  CCT  trainee  performance  and  training  efficiency  were  gathered 
via  carefully  prepared  questionnaires  that  were  completed  by  simulator  in- 
structors, students,  staff  agencies,  and  units  to  which  graduates  were 
assigned.  Based  upon  the  responses  made  to  the  questionnaires,  conclusions 
were  drawn  concerning  the  effectiveness  of  the  LVS  and  the  associated 
training  programs,  including  possible  reduction  in  training  flying  hours 
through  use  of  the  LVS.  While  the  planning  effort  that  went  into  this 

^C-5/C-141  Limited  Visual  System.  Operational  Test  and  Evaluation 
Fina'  Report  5-10-73.  Headquarters,  Military  Airlift  Command,  Scott  AFB, 
TL,  November  1974. 
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study  and  the  use  of  a comprehensive  questionnaire  probably  contributed  to 
the  completeness  and  reliability  of  the  opinion  data  thus  obtained,  the 
basic  deficiency  of  the  Opinion  Survey  Model  as  a means  of  assessing  simula- 
tor training  effectiveness  must  be  considered  in  assessing  this  study's 
conclusions . 

COMMENT 

In  spite  of  the  general  absence  of  formal  evidence  of  the  value  of 
the  Air  Force  simulator  training  activities  surveyed,  a strong  belief  was 
noted  during  the  survey  that  simulator  training  is  a valuable  training 
tool.  Instructors  and  other  personnel  responsible  for  simulator  training 
frequently  expressed  the  belief  that  their  simulator  training  programs 
contribute  significantly  to  the  cost  effective  conduct  of  aircrew  train- 
ing in  both  the  CCT  and  CT  programs  they  administered.  Likewise,  trainees 
typically  expressed  a dependency  upon  simulators  so  far  as  their  progress 
through  the  various  training  programs  was  concerned.  Simulator  training 
clearly  is  an  accepted  part  of  present  day  Air  Force  aircrew  training, 
and  the  personnel  Involved  in  that  training  are  firmly  convinced  of  its 
effectiveness . 

It  may  be  that  simulator  training  has  become  too  well  accepted  by 
the  Air  Force.  The  skepticism  that  characterizes  some  other  training 
programs — a skepticism  that  was  not  widely  noted  during  the  present 
study — serves  a useful  purpose.  It  leads  to  the  requirement  that  any 
simulator  training  activity  be  justified  in  terms  of  its  training  value 
and  cost.  As  a consequence,  some  simulator  training  programs  have  been 
rejected  when  their  value  could  not  be  established,  and  others  have  been 
refined  on  the  basis  of  Information  made  available  during  evaluation 
studies . 

It  is  not  intended  here  to  suggest  that  any  Air  Force  simulator 
training  activities  are  ineffective  and  should  be  discarded.  No  evidence 
was  found  during  the  survey  to  suggest  a single  instance  of  ineffective 
simulator  training.  It  is  possible,  however,  that  a systematic,  objective 
determination  of  the  effectiveness  of  each  of  these  programs  might 
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indicate  their  strengths  and  weaknesses,  and  the  information  thus  obtained 
could  be  used  to  increase  their  effectiveness. 
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III.  PRINCIPAL  FACTORS  INFLUENCING  SIMULATOR 
TRAINING  EFFECTIVENESS 


INTRODUCTION 

The  primary  objective  of  the  re.search  described  in  this  report  was  to 
identify  the  principal  factors  which  influence  the  effectiveness  of  Air 
Force  simulator  training.  The  approach  adopted  in  seeking  to  Identify  these 
factors  was  two-fold:  (a)  to  survey  selected  Air  Force  simulator  training 

activities  in  order  to  identify  factors  operating  upon  or  within  those 
activities  to  influence  their  effectiveness;  and  (b)  to  search  the  simula- 
tor training  research  and  technical  literature  to  Identify  factors  that 
were  likely  Influences  upon  simulator  training.  This  section  of  the  report 
identifies  the  factors  found  during  the  survey  and  literature  search. 

The  Survey 

The  survey  encompassed  the  ten  Air  Force  simulator  training  activities 
identified  in  Section  I.  The  survey  included  both  inspection  of  the  simu- 
lators involved  in  those  activities  and  interviews  with  simulator  instruc- 
tors and  trainees  and  with  ISD  and  supervisory  personnel  responsible  for 
the  development  and  conduct  of  simulator  training  programs.  During  the 
survey,  extensive  Inquiries  were  made  concerning  simulator  training  re- 
quirements, procedures,  and  practices,  and  attempts  were  made  to  relate 
the  responses  obtained  to  the  effectiveness  of  simulator  training  at  each 
of  the  bases  Included  in  the  survey. 

The  evidence  concerning  simulator  training  effectiveness  obtained 
during  the  survey  is  described  earlier  in  this  report.  In  general,  that 
evidence  was  not  such  that  simulator  training  effectiveness  could  be  re- 
lated directly  to  specific  characteristics  of  the  simulator  training 
activities  surveyed.  As  has  been  noted  above,  the  interests  of  Air  Force 
personnel  responsible  for  the  conduct  of  simulator  training  activities 
tend  to  be  primarily  concerned  with  determining  the  effectiveness  of  the 
particular  training  programs  for  which  they  are  responsible  rather  than 
wit'i  the  conduct  of  research  to  investigate  factors  tliat  might  tend  to 
Influence  the  effectiveness  of  their  training  programs. 
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Thus,  while  a number  of  factors  were  identified  during  the  survey  that 
appeared  to  be  at  least  potential  influences  upon  the  effectiveness  of  the 
simulator  training  activities  surveyed,  the  magnitude,  or  even  the  direc- 
tion in  some  cases,  of  their  influence  could  only  be  hypothesized.  Confirm- 
ation that  they  do  in  fact  influence  simulator  training  thus  rests  largely 
upon  Indirect  evidence.  Such  evidence  is  to  be  found  primarily  in  the  research 
and  technical  literature  related  to  simulator  training,  rather  than  in  con- 
trolled studies  of  simulator  training  effectiveness  within  the  operational 
training  activities  surveyed. 

The  Literature  Search 

A summary  of  simulator  training  studies  by  Puig^  indicated  that  simu- 
lator training  effectiveness  has  increased  markedly  since  World  War  II.  Some 
of  the  increase  undoubtedly  can  be  attributed  to  advances  in  engineering 
and  instructional  design  technologies.  Simulation  engineers  now  have  the 
technology  available  to  build  simulators  that  more  nearly  satisfy  Thorndike's 
common  elements  design  hypothesis,  and  instructional  system  designers  have 
learned  how  to  zero  in  on  tasks  to  be  trained.  But,  much  of  the  increase 
has  come  about  as  a result  of  research  and  experience  with  simulators  in 
operational  training  settings.  The  present  project  included  a search  of  the 
literature  that  described  that  simulator  training  research  and  experience. 

In  spite  of  the  large  number  of  simulator  effectiveness  studies  to  be 
found  in  the  research  and  technical  literature,  it  is  evident  that  there 
remains  much  to  be  learned  about  factors  influencing  training  with  these 
devices.  A number  of  factors  have  been  nominated  in  the  literature  as  in- 
fluences upon  simulator  training  effectiveness,  but  many  of  those  nominations 
have  been  based  on  inference  rather  than  experimental  evidence. 

In  their  review  of  simulator  research.  Muckier,  Nygaard,  O'Kelley,  and 
2 

Williams  noted  that  many  studies  compound  the  influence  of  several  potential 

^ Cited  in  Micheli,  G.  Analysis  of  the  Transfer  of  Training,  Substitution 
and  Fidelity  of  Simulation  of  Training  Equipment.  TAEG  Report  2,  Naval  Train- 
ing Equipment  Center,  Orlando,  FE,  1972. 

2 

Muckier,  F.  A.,  Nygaard,  .1.  E. , O'Kelley,  L.  I.  and  Williams,  A.  C.  , Jr., 
Psychological  Variables  in  the  Design  of  Flight  Simulators  for  Training. 
Technical  Report  WAi)C  56-369,  Aerospace  Medical  Laboratory,  Wright-Patterson 
AFB,  OH,  1959. 
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influences,  such  as  training  program  content,  instructional  technique,  and 
instructor  qualification,  into  a single  independent  variable  so  that  realized 
training  benefits  can  be  attributed  only  to  the  unique  combination  of  those 
influences  in  a particular  study.  Even  in  the  few  experimental  investiga- 
tions that  isolate  assumed  influences,  the  results  must  be  interpreted  cau- 
tiously, because  they  address  unique  training  requirements  and  the  studies 
have  seldom  been  replicated. 

Definitive  data  could  not  be  found  in  the  literature  that  would  permit 
the  quantification  of  the  influence  of  all  factors  noted  during  the  survey 
of  Air  Force  training  which  may  be  potential  influences  upon  simulator  train- 
ing effectiveness.  In  fact,  the  mere  identification  of  most  such  factors 
rests  upon  Inference,  conjecture,  and  untested  hypotheses.  The  absence  of 
hard  data  obviously  cannot  justify  suspected  factors  being  Ignored,  however. 
Where  inferences  can  be  made  and  supported  by  consensus,  factors  believed 
to  influence  simulator  training  must  be  taken  into  account  by  those  res- 
ponsible for  simulator  design  and  use,  unless  evidence  can  be  assembled  to 
refute  those  inferences. 

Methodological  Problem s 

The  methodological  problems  involved  in  identifying  factors  that  in- 
fluence simulator  training  effectiveness  cannot  be  overcome  easily.  Suspected 
factors  can  seldom  be  examined  in  isolation.  It  is  difficult,  for  example, 
to  determine  experimentally  the  relative  value  of  a remote  simulator  in- 
structor station  vs.  an  on-board  or  in-the-cockpit  station,  even  if  a suit- 
ably designed  simulator  were  available  for  the  researcli,  because  to  use 
each  station  to  its  best  advantage  would  necessitate  having  two  methods  of 
training:  one  optimized  for  remoted  instruction;  the  other  optimized  for 

on-board  instruction.  The  experiment  would  thus  compare  instructor  station- 
training program  combinations,  not  a simulator  design  feature  in  isolation 
from  other  factors.  The  training  program  factor  cannot  be  held  constant.  It 
would  be  Inappropriate  to  compare  two  simulator  designs  using  a program 
optimized  for  only  one,  or  for  neither. 

The  problem  of  general izable  results  is  not  limited  to  studies  involving 
operational  simulators.  Even  using  equipment  designed  and  dedicated  to 
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research,  problems  arise.  For  example,  to  pursue  the  illustration  of  in- 
structor station  location  described  above,  simulator  hardware  inflexibility 
makes  it  difficult  to  conduct  the  necessary  research  leading  to  the  design 
of  the  optimum  remote  instructor  station  for  experimental  comparison  with 
the  optimally  designed  on-board  station.  Addi tional ly , the  on-board  station 
design  that  is  optimum  for  a single  seat,  high  performance  attack  aircraft 
simulator  with  a visual  display  may  bear  little  resemblance  to  the  optimized 
on-board  station  for  undergraduate  instrument  training  in  a side-by-side 
seating  helicopter  simulator. 

FINDINGS  FROM  THE  SURVEY  AND  LITERATURE  SE/\RCH 

The  following  discussion  summarizes  the  information  about  factors  in- 
fluencing simulator  training  obtained  during  the  inspection  of  Air  Force 
simulators  and  the  Interviews  with  personnel  associated  with  their  use. 

The  various  factors  observed  by  the  writer  and/or  suggested  by  the  per- 
sonnel interviewed  are  described.  In  addition,  relevant  information  from 
the  research  and  technical  literature  is  presented  in  an  attempt  to  assess 
the  probable  influence  of  the  factors  cited  upon  the  effectiveness  of 
simulator  training  in  the  Air  Force. 

This  information  is  organized  according  to  subject  matter,  i.e., 
simulator  design,  training  programs,  personnel,  attitudes,  expectations, 
and  other  miscellaneous  factors.  This  organization  was  adopted  as  a con- 
venient vehicle  to  portray  the  writer's  impressions.  Decisions  to  identify 
a factor  as  associated  with  a particular  subject  matter  area  were  some- 
times made  arbitrarily.  The  adopted  organization  provides  a convenient 
conceptual  framework  wltiiin  which  to  structure  a discussion  of  factors 
influencing  simulator  training  effectiveness,  but  it  by  no  means  is  intended 
to  impose  constraints  upon  the  identification  of  other  factors  that  might 
be  considered  by  others  to  influence  simulator  training  effectiveness. 

Simulator  Design 

There  are  two  areas  of  Interest  with  respect  to  the  influence  of  simu- 
lator design  upon  transfer  of  training:  fidelity  of  simulation  and  design 

for  training.  Fidelity  refers  to  whether  features  of  the  aircraft  and  its 
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envlionment  are  Included  in  the  simulator's  design,  and  the  extent  to  which 
features  that  are  included  represent  or  duplicate  their  real  world  counter- 
parts. Design  for  training  refers  to  the  inclusion  in  simulator  design  of 
features  or  configurations  that  facilitate  training,  but  that  may  bear  no 
particular  resemblance  to  features  of  the  aircraft  and  environment  being 
simulated . 

Fidelity  of  simulation  is  often  equated  with  physical  correspondence 
between  the  device  and  its  real  world  counterpart.  In  a discussion  of  simu- 
lator design  considerations,  however,  Smode  and  Hall^  emphasize  instructional 
strategies  and  capabilities  and  suggest  that  fidelity  has  meaning  in  terms 
of  the  process  and  the  realism  necessary  to  promote  learning.  Design  char- 
acteristics, they  assert,  should  be  defined  in  terms  of  assuring  transfer 
of  training.  In  other  words,  fidelity  of  simulation  is  a matter  of  the 

relevance  of  the  simulation  to  the  training  objectives,  not  solely  a matter 

2 

of  physical  correspondence.  Bryan  and  Regan  also  noted  this  relationship 
between  fidelity  and  training  when  they  suggested  that  simulator  design  can 
omit  features  of  the  aircraft,  even  cockpit  components  with  which  the  pilot 
might  normally  interact,  if  the  features  are  not  part  of  the  subject  matter 
of  the  intended  training.  Tliis  concept  of  fidelity  accounts  for  the  reported 
training  effectiveness  of  so-called  low  fidelity  devices,  such  as  those  made 
of  plywood  and  photographs,'^  as  well  as  simulators  that  faithfully  reproduce 
much  of  the  aircraft. 

Fidelity  of  simulation  was  cited  as  a factor  that  influenced  the  effect- 
iveness of  several  of  the  simulator  training  programs  included  in  the  present 

^Smode,  A.  F.  and  Hall,  E.  R.  Translating  Information  Requirements  into 
Training  Device  Fidelity  Requirements.  In  Proceedings  Human  Factors  Society 
19th  Annual  Meeting.  Human  Factors  Society,  Santa  Monica,  CA,  1975. 

2 

Bryan,  G.  b.  and  Regan,  J.  J.  Training  System  Design.  In  Van  Cott, 

H.  P.  and  Kinkade,  R.  G.  (Ed.),  Human  Engineering  Guide  to  Equipment  Design 
(Rev.  ed.).  Wasiiington:  Government  Printing  Office,  1972. 

3 

Prophet,  W.  W.  and  Boyd,  H.  A.  Device-Task  Fidelity  and  Transfer  of 
Training:  Aircraft  Cockpit  Proced^iirj^  Tra ininj^ . Techniv"al  Report  70-10, 

Human  Resources  Researcii  Organization,  Alexandria,  VA,  Jury  1970. 
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survey.  Some  of  these  simulators,  for  example,  had  not  been  modified  to 
reflect  all  of  the  changes  that  have  occurred  In  the  design  model  aircraft. 
This  was  found  to  be  particularly  true  with  respect  to  avionics  and  EW/ECM 
systems.  The  result  has  been  to  reduce  the  physical  correspondence  between 
the  simulator  and  the  aircraft  to  the  point,  in  some  instances,  tliat  train- 
ing required  to  accomplish  a mission  in  the  aircraft  cannot  be  conducted  in 
the  simulator.  It  was  noted  in  most  such  instances,  however,  that  steps 
were  being  undertaken  to  correct  these  discrepancies,  and  tlius  to  increase 
the  range  of  training  activities  that  can  be  undertaken  in  the  device. 
Nevertheless,  the  almost  inevitable  lag  between  the  time  the  aircraft  is 
modified  and  the  time  that  that  modification  is  incorporated  into  the  simu- 
lator appears  to  affect  simulator  training  effectiveness  in  two  ways:  certain 

performances  required  in  the  aircraft  cannot  be  trained  in  the  simulator; 
and  differences  between  the  device  and  the  aircraft  detract  from  simulator 
training  and  tend  to  reduce  its  perceived  value,  particularly  among  trainees 
who  may  have  unfavorable  attitudes  toward  simulation  anyway. 

Design  for  Training:  I'he  Smode  and  Hall  concept  of  fidelity  is  of  particular 

interest  with  respect  to  simulator  features  not  modeled  after  the  aircraft. 
These  features  that  are  concerned  primarily  with  application  of  principles 
of  learning  to  the  training  process  include  freeze,  adaptive  training, 
prompting  and  cueing,  performance  recording  and  playback,  performance  measure- 
ment, and  various  instructor  station  displays  and  controls.  It  is  generally 
held  that  such  features  improve  the  conditions  under  which  learning  takes 
place  and  thereby  facilitate  the  attainment  of  training  objectives.^  There- 
fore, they  are  factors  to  be  considered  in  judging  the  fidelity  of  a device 
so  far  as  training  is  concerned. 

It  is  general  practice  to  adopt  innovative  simulator  design  features 
such  as  those  mentioned  above  on  the  basis  of  their  apparent  utility  with- 
out subjecting  them  to  exnerimental  scrutiny.  For  example,  the  widely  used 
simulator  freeze  feature  was  implemented  because  it  was  seen  as  an  aid  to 
attaining  training  objectives  and  to  implementing  learning  concepts  during 
the  instructional  rr'cess.  Similarly,  other  design  decisions  are  made 
because  the  training  objectives  and  planned  concepts  of  simulator  employment 

^Prophet,  W.  W. , Caro,  P.  W.  and  Hall,  E.  R.  Some  Current  Issues  in 
the  Design  of  Training  Devices,  25^_h  Armive_rsa^_Commemora^i”c  Technical 
-lournal . Naval  Training  Equipment  Center,  Orlando,  FE,  November  1971. 
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lead  to  the  analytical  conclusion  that  a particular  design  is  appropriate 
in  preference  to  others.  For  example,  in  the  design  of  the  U.  S.  Air  Force 
simulators  for  the  H-3  and  H-53  helicopters  and  the  U.  S.  Army  simulators 
for  the  CH-47  and  the  AH-1  helicopters,  the  instructor  stations  were  located 
virtually  inside  the  cockpits  of  these  devices,  and  certain  instructor  dis- 
plays were  positioned  so  that  they  could  be  viewed  by  both  the  instructor 
and  the  trainees  in  order  to  facilitate  instructor-trainee  interactions 
during  key  training  activities.  The  training  effectiveness  of  these  features 
probably  will  never  be  determined  in  a transfer  experiment  for  reasons  dis- 
cussed above.  Analytically,  they  are  believed  to  represent  effective  simu- 
lator designs  with  respect  to  the  training  programs  and  the  training  ob- 
jectives being  addressed. 

The  design  of  simulators  for  the  FB-111  at  Plattsburgh  Air  Force  Base 
provides  an  opportunity  for  an  indirect  examination  of  the  question  of 
instructor  position  location.  The  FB-111  Aircraft  Simulator  consists  of  a 
replication  of  the  positions  of  the  pilot  and  the  bombardier/navigator  in 
the  cockpit  of  the  aircraft,  these  two  positions  being  mounted  on  a five- 
axis  motion  platform.  Positions  for  a pilot  Instructor  and  for  a bombardier/ 
navigator  Instructor  are  provided  off  the  motion  platform.  In  addition 
to  this  de'.'lce,  there  is  a separate  FB-111  Bomb/Nav  Simulator  consisting  of 
a replication  of  the  bombardier/navigator's  cockpit  position  with  an  adja- 
cent instructor  position,  neither  of  which  is  mounted  on  a motion  platform. 
The  latter  device  is  intended  primarily  as  a part-task  trainer  in  which  the 
bomb/nav  tasks  are  emphasized,  wiiereas  the  larger  simulator  is  intended  for 
full-mission  training  Involving  both  FB-111  crew  members.  A major  portion 
of  the  mission  tasks  of  the  bombardier/navigator  can  be  simulated  in  either 
device,  however.  In  the  part-task  simulator  that  lacks  a pilot  position, 
most  pilot  tasks  can  be  "simulated"  by  the  instructor,  whose  position  in 
the  device  approximates  the  position  of  the  pilot  in  the  aircraft.  Thus 
the  training  program  and  the  role  and  position  of  the  bombardier/navigator 
instructor  in  these  two  devices  are  quite  different. 

The  limited  resources  available  during  the  present  survey  did  not  permit 
a comparison  to  be  made  of  the  effectiveness  of  bombardier/navigator  train- 
ing in  these  two  devices.  Nevertheless,  several  comments  were  made  by 
Instructors  at  Plattsburgh  favoring  the  on-board  instructor  position.  The 
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bombardier/navigator  instructors  interviewed,  each  of  whom  conducted  training 
in  both  devices,  indicated  that,  in  their  judgment,  they  were  able  to  instruct 
>■  more  effectively  in  the  Bomb/Nav  Simulator  because  they  could  see  what  the 

trainee  was  doing  and  could  diagnose  his  difficulties  more  easily  than  in  the 
other  device.  The  trainees  also  expressed  a preference  for  the  Bomb/Nav 
Simulator,  commenting  that,  in  it,  the  instructor  was  there  to  help,  whereas 
in  the  larger  simulator,  the  instructor  was  "out  there  plugging  in  malfunc- 
tions." UTiile  factors  other  than  simulator  design  could  be  operating  within 
the  small  sample  interviewed  and  might  account  for  such  comments,  the  existence 
of  these  two  devices  presents  an  opportunity  for  further  examination  of  this 
major  aspect  of  simulator  design. 

The  design  of  the  Air  Force's  C-5  simulator  is  such  that  most  of  the 
instructing  is  done  by  on-board  instructors  seated  near  the  trainees  of  interest, 
i.e.,  pilot  instructor,  flight  engineer  instructor,  and  navigator  instructor. 

The  C-5  simulator  also  has  a remote  position  from  which  an  additional  instructor 
can  provide  minor  assistance  to  the  on-board  instructors  related  to  problem 
set-ups  and  can  provide  voice  communications  from  simulated  ground  stations. 
Instruction  cannot  be  conducted  from  this  remote  position,  however.  Comments 
were  made  by  the  on-board  instructors  about  the  role  of  the  remote  Instructor 
that  suggested  that  the  latter's  performance  was  not  always  of  the  highest 
quality.  Observation  of  the  remote  instructor  during  a simulator  training 
mission  suggested  a possible  reason  for  the  reported  quality  of  his  performance 
that  relates  to  simulator  design;  he  has  too  little  to  do  to  maintain  his 
attention.  For  the  most  part,  he  has  been  designed  out  of  the  instructional 
process.  Consequently,  he  tends  to  become  inattentive  and  thus  is  prone  to  make 
errors . 

In  the  simulator  for  the  F-lOb  at  Castle  Air  Force  Base,  where  the  instruc- 
tor positi'on  is  beside  the  cockpit  (no  platform  motion  is  involved),  the  instruc- 
tors expressed  a preference  for  instructing  with  the  canopy  open.  It  was  their 
feeling  that,  with  the  Continuation  Training  student,  the  instructor  could  be 
more  effective  when  his  Interactions  with  the  student  Involved  direct  observa- 
tion. They  reported  that  with  the  canopy  open,  their  interactions  with  the 
trainees  were  more  frequent  also.  It  should  be  noted  that  similar  use  of  the 
same  simulator  In  Combat  Crew  Training  at  Tyndall  Air  Force  Base  was  not 
reported,  and  the  Instructors  there  felt  that  more  effective  training  could 
be  provided  when  the  canopy  was  closed,  as  it  is  in  the  aircraft  in  flight. 
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The  proximity  of  the  F-106  simulator  cockpit  to  the  device's  instructor 
>,  console  permits  the  simulator  instructor  to  interact  with  the  trainee  when 

the  canopy  is  open  while  maintaining  access  to  his  controls  and  displays. 

With  simulators  where  such  proximity  does  not  exist,  such  as  the  F-4  and  A-7 
simulators,  very  little  instruction  reportedly  takes  place  during  CCT  or  CT 
with  the  canopy  open.  The  instructor  using  these  latter  devices  occasionally 
gits  in  a fold-down  jump  seat  from  which  he  can  observe  the  trainees  directly, 
but  the  motion  platform  must  be  off  when  the  jump  seat  is  occupied,  and  there 
are  very  few  controls  available  there  for  instructor  use.  Other  differences 
between  F-106  and  other  simulator  training  that  undoubtedly  influence  the 
extent  to  which  the  training  is  conducted  with  the  canopy  open  include  the 
training  syllabi  and  the  availability  of  other  devices  such  as  CPTs  in  which 
portions  of  the  training  that  would  benefit  most  from  observation  of  trainee 
responses  can  be  conducted. 

The  construction  of  separate  simulators  for  each  crew  station  of  a 
multi-crew  aircraft,  as  opposed  to  construction  of  a single  simulator  with 
each  crew  station  represented,  was  noted  as  a likely  influence  upon  simulator 
training  effectiveness.  While  it  makes  possible  more  efficient  training  of 
individual  crew  members,  in  some  Instances  separate  simulators  may  make 
training  in  tasks  requiring  the  coordinated  activity  of  more  than  one  crew- 
man more  difficult. 

Visual  Fidelity.  Tasks  that  cannot  bo  duplicated  or  even  approximated  in  a 
device  cannot  be  learned  there  for  subsequent  transfer  to  the  aircraft. 
Therefore,  a simulator  in  which  more  tasks  characterizing  flying  can  be  per- 
formed has  greater  potential  training  effectiveness  than  one  in  which  fewer 
such  tasks  can  be  performed.  For  example,  a simulator  that  Includes  an 
extra-cockpit  visual  display  would  seem  to  have  more  effectiveness  potential 
with  respect  to  training  tasks  requiring  visual  references  than  a simulator 
without  such  a display. 

There  has  been  a number  of  simulator  training  studies  involving  visual 
displays  in  which  transfer  of  visual  flight  skills  has  been  demonstrated. 

The  scenes  presented  by  the  displays  used  in  some  of  these  studies  are  much 
simpler  than  scenes  viewed  from  an  aircraft.  For  example,  savings  in  air- 
craft time  required  to  perform  visual  reference  maneuvers  were  demonstrated 
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by  Flexman,  Matheny  and  Brown  in  a study  using  a simulator  with  a very 

simple  visual  display.  The  display  consisted  of  a line  drawing  of  a runway 

on  a blackboard  that  was  placed  in  front  of  the  device's  cockpit  and  tilted 

by  an  instructor  to  make  the  runway  change  perspective  as  the  device  was 

maneuvered  with  respect  to  simulated  ground  references.  The  effectiveness 

of  displays  consisting  of  stylized  grids  and  lines  has  been  demonstrated 

in  backward  transfer  situations  during  studies  of  contact  analog  displays 

2 

developed  for  helicopters.  Displacement  of  visual  scene  elements  consist- 

3 

ing  only  of  dots  and  lines  was  found  by  Thielges  and  Matheny  to  provide 
sufficient  information  for  the  performance  of  basic  aircraft  control  tasks. 
Studies  such  as  these  indicate  that  tasks  involving  aircraft  control  in  re- 
lation to  extra-cockpit  visual  information  can  be  practiced  effectively  in 
simulators  with  very  simple  visual  scene  displays.  Such  displays  can  con- 
sist of  no  more  than  points,  lines  and  geometric  patterns  arranged  in  accord 

4 

with  a set  of  mathematical  relationships  first  described  by  Gibson. 

Several  manufacturers  currently  are  marketing  displays  that  employ 
these  simple  scene  content  design  concepts.  Their  displays  represent  night 
scenes  as  patterned  points  of  light  on  a dark  field.  These  night  only  visual 
displays  have  considerable  face  validity  and  have  been  well  received  for  use 
in  both  commercial  airlines  and  military  simulator  training  programs.  While 
part  of  their  acceptance  by  pilots  might  be  attributed  to  the  relatively 
large  amount  of  information  that  has  been  represented  by  the  light  patterns, 
the  use  of  color,  and  the  addition  of  textured  surfaces  in  some  instances, 
the  research  literature  clearly  suggests  that  even  the  simplest  of  these 
displays  have  training  value. 

^Flexman,  R.  E.,  Matheny,  W.  G.  and  Brown,  E.  G.  Evaluation  of  the 
School  Link  and  Special  Methods  of  Instruction  in  a Ten-Hour  Private  Pilot 
Flight-Training  Program.  University  of  Illinois  Aeronautics  Bulletin  No. 

8,  Urbana,  IL,  1950. 
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The  effectiveness  of  such  simple  visual  displays  has  been  demonstrated 
principally  with  respect  to  basic  aircraft  control  and  approach  and  landing 
maneuvers.  It  is  also  noted,  however,  that  simulators  without  an  extra- 
cockpit visual  display  can  be  effective  in  the  training  of  similar  visual 
reference  flight  tasks.  For  example,  in  a study  involving  a helicopter  simu- 
lator without  a visual  display  or  any  other  representation  of  outside  visual 
cues  except  the  aircraft's  navigation  and  attitude  instruments,  and  without 
any  attention  during  simulator  training  to  extra-cockpit  visual  cues  per  se, 
studevics  trained  to  fly  instrument  flight  missions  in  the  device  qualified 
in  the  aircraft  under  visual  conditions  more  rapidly  than  did  students  not 
receiving  the  prior  device  training.^  In  a similar  study  using  a fixed  wing 
simulator  without  a visual  display,  a saving  in  visual  flight  time  required 

to  complete  a transition  course  of  approximately  50%  of  the  scheduled  course 
2 

length  was  obtained.  The  successful  use  of  the  non-visually  equipped  A-7D 
simulator  at  Davls-Monthan  Air  Force  Base  to  prepare  CCT  students  for  the 
first  visual  solo  flight  in  the  aircraft  was  mentioned  in  an  earlier  section 
of  this  report. 

The  transfer  of  skills  acquired  in  a non-visually  equipped  simulator  to 
an  aircraft  is  consistent  with  findings  of  research  involving  the  conduct  of 
instrument  flight  training  in  aircraft  prior  to  visual  flight  training. 

These  so-called  "instruments  first"  studies  have  consistently  demonstrated 
that  instrument  flight  skills  facilitate  the  acquisition  of  visual  flight 
skills.  The  substitution  of  a simulator  for  an  aircraft  as  the  locus  for 
instrument  training  should  be  expected  to  result  in  similar  training  benefits. 

It  is  not  indicated  by  these  studies'  results  that  visual  displays  have 
no  training  value,  that  all  visual  discriminations  pilots  need  can  be 
learned  in  simulators  without  visual  displays,  or  even  that  any  visual 
flight  skills  can  be  fully  developed  in  a simulator  without  a visual  dis- 
play or  practice  of  visual  tasks  in  an  aircraft.  They  do  suggest,  however, 

^Caro,  P.  W. , Isley,  R.  N.  and  Jolley,  0.  B.  Mission  Suitability 
Testing  of  an  Aircraft  Simulator.  Technical  Report  75-12,  Human  Resources 
Research  Organization,  Alexandria,  VA,  June  1975. 
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Caro,  P.  W. , Isley,  R.  N.  and  Jolley,  0.  B.  Research  on  Synthetic 
Training;  Device  Evaluation  and  Training  Program  Development.  Technical 
Report  73-20,  Human  Resources  Research  Organization,  Alexandria,  VA,  1973. 
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that  many  tasks  performed  in  response  to  extra-cockpit  visual  stimuli  in 
the  aircraft  can  be  learned,  at  least  in  part,  through  practice  in  response 
to  stimuli  in  a simulator  without  an  outside  display.  Conventional  cockpit 
Instruments  provide  an  analog  display  of  the  visual  world  outside  the  cock- 
pit, so  a pilot  flying  instruments  is  responding  to  stimuli  analogous  to 
those  available  to  the  pilot  flying  visually.  Similarly,  single-window 
computer  generated  visual  scene  displays  are  analog  displays  which  are 
viewed  by  the  pilot  trainee  through  the  simulator  wind  screen.  An  instrument 
panel  display  that  presented  the  same  information  inside  the  cockpit  instead 
of  through  the  wind  screen  probably  would  be  equally  effective  from  the 
training  standpoint. 

Even  in  those  situations  in  which  necessary  cues  cannot  be  represented 
adequately  by  an  instrument  or  analog  display,  alternatives  may  exist  to  the 
provision  of  an  extra-cockpit  visual  display  for  some  training.  For  example, 
the  Air  Force  has  demonstrated  significant  improvement  in  the  performance  of 
visual  flight  maneuvers  following  training  in  a classroom  using  35  mm  slides 
and  8 mm  movies.^  Wliile  the  scope  of  the  training  involved  in  this  line  of 
research  has  been  limited,  it  does  suggest  that  visual  displays  can  be  sep- 
arated from  the  simulator  altogether  for  at  least  a portion  of  the  training 
required  to  meet  some  training  objectives. 

While  there  have  been  numerous  studies  of  the  training  effectiveness  of 
simulators  with  extra-cockpit  visual  displays,  few  of  those  studies  included 
a control  group  in  which  students  were  trained  in  the  simulator  without 
using  its  visual  display.  In  the  only  study  with  such  a control  group 
known  to  the  present  writer  that  examined  pilot  performance  in  the  aircraft, 

following  training  in  a simulator,  the  results  were  judged  inconclusive  by 

2 3 

its  authors.  DeBerg,  McFarland,  and  Showalter,  in  a study  based  upon 

the  Simulator-to-Simulator  Transfer  Model  described  earlier,  examined  the 

^Smith,  B.  A.,  Waters,  B.  K.  and  Edwards,  B.  J.  Cognitive  Pretraining 
of  the  T-37  Overhead  Traffic  Pattern.  Technical  Report  AFHRL-TR-75-72,  Air 
Force  Human  Resources  Laboratory,  Williams  AFB,  AZ,  December  1975. 
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effects  upon  trainlag  of  adding  a visual  display  to  an  existing  simulator 
during  the  simulation  of  engine  failure  during  takeoff.  Their  data  show 
an  improvement  in  post-training  pilot  performance  when  the  visual  display 
was  used  and  the  simulator's  motion  system  was  employed,  but  pilot  perform- 
ance when  only  the  visual  (i.e.,  no  motion)  was  used  was  inferior  to  per- 
formance when  neither  visual  nor  motion  was  employed. 

Commercial  airlines  have  reduced  aircraft  training  time  following  the 
addition  of  a visual  display  to  an  existing  simulator,  but  some  if  not  all 
of  such  reductions  resulted  from  a priori  judgments  by  government  agencies, 
pilot's  associations  and  the  airlines  themselves  concerning  increased  simu- 
lator training  effectiveness.  Likewise,  the  amount  of  flight  training 
included  in  the  C-5A^  CCT  programs  at  Altus  Air  Force  Base  was  reduced 
following  the  addition  of  a visual  display  to  that  simulator,  but  the  amount 
of  the  reduction  was  based  upon  a priori  judgments,  later  confirmed  by  post 
hoc  opinions,  rather  than  by  an  empirical  determination  of  the  increased 
value  of  simulator  training  following  addition  of  the  visual  display.  The 
assumption  that  extra-cockpit  visual  displays  increase  simulator  training 
effectiveness  are  so  strong  that  their  value  is  seldom  questioned.  Valida- 
tion of  their  training  benefits  has  not  been  sought  in  most  cases. 

A lack  of  empirical  evidence  of  the  training  effectiveness  of  visual 
displays  cannot  be  taken  as  evidence  of  their  lack  of  effectiveness.  There 
is  a consensus  that  they  are  effective,  particularly  for  the  more  complex 
visual  tasks  associated  with  flying,  and  data  to  contest  that  consensus  do 
not  exist.  Logically,  it  would  appear  that  an  extra-cockpit  visual  display 
is  an  effective  way,  and  possibly  the  only  way  in  some  Instances,  to  present 
visual  information  needed  for  many  operational  tasks,  e.g.,  formation  flight, 
refueling,  delivering  certain  kinds  of  weapons,  and  air-to-air  combat.  For 
others,  e.g.,  taxiing,  flying  traffic  patterns,  making  visual  approaches,  and 
performing  basic  aircraft  control  maneuvers,  it  may  be  effective  but  inef- 
ficient, particularly  when  cost  is  taken  into  consideration. 

Regardless  of  their  demonstrated  effectiveness  (or  lack  thereof),  it 
has  been  observed  that  commercial  pilots  prefer  to  train  in  simulators  that 

^The  C-5  simulator  at  Altus  Air  Force  Base  was  the  only  simulator 
included  in  the  current  survey  with  a visual  display.  The  evaluation  of  the 
effectiveness  of  training  with  this  device  is  discussed  in  Section  II. 
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have  visual  displays,  and  further,  that  they  prefer  simulators  with  displays 
having  a rich  scene  content,  such  as  that  provided  by  motion  picture  or  TV 
images  of  the  world  as  viewed  from  the  aircraft,  as  opposed  to  displays 
composed  of  points  of  light  on  a dark  field.  ^ Likewise,  the  Air  Force  pilots 
interviewed  during  the  current  survey  were  in  agreement  that  the  effectiveness 
of  the  non-visually  equipped  simulators  in  which  they  train  would  be  increased 
by  the  addition  of  visual  displays.  For  the  most  part,  these  pilots  had  not 
had  the  opportunity  to  inspect  a variety  of  kinds  of  displays,  so  they  held 
no  particular  preference  for  one  kind  over  another.  Since  increased  fidel- 
ity was  frequently  cited  as  the  basis  for  their  desire  for  visual  displays 
in  the  first  place,  however,  it  is  likely  that  they  would  share  with  commer- 
cial pilots  the  preference  for  displays  having  a rich  scene  content.  If 
these  preferences  were  reflected  in  their  willingness  to  undergo  training  in 
simulators  having  such  displays,  as  opposed  to  their  unwillingness  otherwise, 
then  the  presence  of  a visual  display  and  the  richness  of  its  scene  content 
can  Influence  simulator  training  effectiveness. 

Apart  from  the  preferences  of  pilots  regarding  display  richness,  and 
the  possible  consequences  of  such  preferences  upon  simulator  use,  the  train- 
ing research  literature  suggests  strongly  that  quite  simple  visual  displays 
may  have  as  much  training  value  as  those  rich  in  scene  content.  Relevant 
information  must  be  displayed,  of  course,  but  it  sometimes  appears  that 
unnecessary  emphasis  is  placed  upon  designing  visual  displays  that  look  like 
the  "real  world"  viewed  through  the  aircraft's  wind  screen.  There  may  be 
a need  for  such  a level  of  scene  fidelity  with  respect  to  some  training 
requirements,  but  more  often,  it  is  likely  that  the  display  design  require- 
ments are  more  responsive  to  fidelity  goals  than  to  training  necessities. 

In  designing  visual  displays  for  simulators,  it  would  be  less  costly  to  seek 

simple  displays  whenever  simple  ones  are  responsive  to  well  defined  training 

2 

requirements.  Smode's  and  Hall's  assertion  that  fidelity  of  simulation  is 
a matter  of  the  relevance  of  the  simulation  to  the  training  objective  applies 
to  the  design  of  visual  displays  as  well  as  to  other  simulator  components. 

^Personal  communication,  R.  A.  Waldrop,  American  Airlines,  1975. 
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Smode,  A.  F.  and  Hall,  F. . R.  Translating  Information  Requirements  into 
Training  Device  Fidelity  Requirements.  In  Proceedings  Human  Factors  Society 
19th  Annual  Meeting.  Human  Factors  Society,  Santa  Monica,  CA,  1975. 
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Motion  Fidelity.  Although  motion  simulation  represents  a significant  portion 
of  the  cost  of  simulator  procurement  and  operation,^  the  investigation  of 
the  influence  of  motion  upon  transfer  of  simulator  training  to  operational 
aircraft  has  been  largely  ignored.  There  were  a number  of  studies  of  simu- 
lator motion  in  relation  to  aircraft  handling  qualities  and  control  during 
the  1950s  and  1960s,  but  most  of  them  addressed  transfer  of  training  only 
indirectly.  The  first  significant  published  transfer  of  training  study 

of  the  effectiveness  of  simulator  motion  upon  subsequent  performance  in 

2 

flight  was  reported  in  1975  by  Jacobs  and  Roscoe. 

Jacobs  and  Roscoe  reported  that  pilot  performance  in  the  aircraiu  did 
not  benefit  from  the  presence  of  normal  washout  cockpit  motion  in  the  simu- 
lator. In  that  study,  training  received  in  the  GAT-2  in  a two-axis  (pitch 
and  roll)  normal  washout  motion  condition,  compared  with  training  in  the  same 
device  without  motion,  resulted  in  non-significant  differences  in  amount  of 
transfer  to  the  aircraft  for  those  two  conditions.  There  was,  however, 
significant  positive  transfer  for  both  motion  and  no-raotion  conditions. 
Similar  results  have  been  obtained  in  a U.S.  Air  Force  undergraduate  pilot 

training  study  involving  the  more  sophisticated  six-axis  motion  system 

3 

associated  with  the  Advanced  Simulator  for  Pilot  Training  (ASPT) . 

The  findings  in  these  two  recent  studies  that  the  presence  of  motion 
did  not  Increase  simulator  training  effectiveness  is  of  considerable  inter- 
est, since  there  are  other  studies  showing  that,  at  least  under  some  circum- 

4 

stances,  motion  does  influence  simulator  training.  For  example,  Fedderson 
reported  a slight  advantage  in  favor  of  a motion  simulator  trained 

^Cohen,  E.  Is  Motion  Needed  in  Flight  Simulators  Used  for  Training? 
Human  Factors,  1970,  \1,  75-79. 
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Jacobs,  R.  S.  and  Roscoe,  S.  N.  Simulator  Cockpit  Motion  and  the 
Transfer  of  Initial  Flight  Training.  In  Proceedings  Human  Factors  Society 
19th  Annual  Meeting.  Human  Factors  Society,  Santa  Monica,  CA,  1975. 
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Wood r u f f , R . R . Full  Mission  Simulation  in  Undergraduate  Pilot 
Training:  An  Exploratory  Study,  Technical  Report  AFHRL-TR-76-84 , Air 

Force  Human  Resources  Laboratory,  Williams  AFB,  AZ,  December,  1976. 
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Fedderson,  W.  E.  The  Role  of  Motion  Information  and  Its  Contribution 
to  Simulation  Validity.  Technical  Data  Report  No.  D228-429-001  , Bell 
Helicopter  Company,  Fort  Worth,  TX,  1962. 
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group  over  a no-motion  group  during  brief  transfer  trials  hovering  a heli- 
copter. More  importantly,  perhaps,  the  motion  group  in  his  study  reached 
asymptotic  performance  in  the  simulator  more  rapidly,  suggesting  that  simu- 
lators with  motion  may  provide  more  efficient  training.  A recent  L'.S.  Air 
Force  study  of  pilot  responses  to  engine  failure  in  a simulated  transport- 
type  aircraft  found  that  training  is  more  effective  when  motion  is  added  to 
a simulator  with  a visual  display  than  when  the  same  simulator  and  visual 
are  used  without  motion.^ 

Further,  there  is  evidence  that  pilot  performance  differs  as  a function 

2 

of  the  presence  or  absence  of  motion.  For  example,  Perry  and  Naish  found 

that  pilots  respond  to  external  forcing  functions  such  as  side  gusts  more 

rapidly,  with  more  authority,  and  in  a more  precise  manner  in  a simulator 

with  motion  and  visual  cues  than  when  only  visual  cues  are  present.  NASA 
3 

researchers  found  that  the  correlation  between  pilot  performance  in  an 
aircraft  and  in  a simulator  Increased  with  the  addition  of  simulator  motion 
cues  where  such  cues  help  the  pilot  in  coping  with  a highly  damped  or  un- 
stable vehicle  or  a sluggish  control  system,  or  under  some  circumstances, 
where  the  control  system  is  too  sensitive.  Where  the  aircraft  is  easy  to 
fly,  however,  as  is  the  case  with  the  aircraft  used  in  the  Jacobs  and  Ros- 

coe  study  (Piper  Cherokee)  and  in  the  Air  Force  ASPT  study  (T-37),  motion 

4 

may  have  no  effect.  In  another  NASA  study  of  the  effects  of  simulator 

^DeBerg,  0.  H.,  McFarland,  B.  P.  and  Showalter,  T.  W.  The  Effect  of 
Simulator  Fidelity  on  Engine  Failure  Training  in  the  KC-135  Aircraft. 

Paper  presented  at  American  Institute  of  Aeronautics  and  Astronautics  Visual 
and  Motion  Simulation  Conference,  Dayton,  OH,  April  26-28,  1976. 
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Perry,  D.  H.  and  Naish,  J.  M,  Flight  Simulation  for  Research, 

Journal  of  the  Royal  Aeronautical  Society,  1964,  645-662. 
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Rathert,  G.  A.,  Jr.,  Creer,  B.  Y.  and  Sadoff,  M.  The  Use  of  Piloted 
Flight  Simulators  in  General  Research.  Report  365,  Advisory  Group  for  Aero- 
nautical Research  and  Development,  North  Atlantic  Treaty  Organization,  Paris, 
France,  1961. 
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Douvlllier,  J.  G. , Jr.  Turner,  H.  L. , McLean,  J.  D.  and  Heinle,  D.  R. 
Effects  of  Flight  Simulator  Motion  on  Pilots'  Performance  of  Tracking  Tasks. 
Technical  Note  NASA-TN-D- 143 , National  Aeronautics  and  Space  Administration, 
Washington,  DC,  I960. 
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motion  on  pilot's  performance  of  flight  tracking  tasks,  the  results  from  a 

moving  base  flight  simulator  resembled  the  results  from  flight  much  more 

than  did  those  from  a motionless  simulator.  In  a British  study,  Huddleston 

and  Rolfe^  reported  that  the  presence  of  simulator  motion  produced  patterns 

of  control  response  more  closely  related  to  those  employed  in  flight.  That 

is,  using  simulators  without  motion,  experienced  pilots  were  able  to  achieve 

acceptable  levels  of  performance,  but  their  patterns  of  control  response 

showed  that  their  performance  was  achieved  using  a strategy  different  from 

that  used  in  a dynamic  training  environment.  Research  at  the  University 

of  Illinois  related  to  instrument  display  design  found  that  the  quality  of 

the  simulator  motion  involved  affected  pilot  responses  to  display  types 

differentially,  with  inappropriate  banking  motions  interfering  with  command 

2 

flight  path  tracking. 

Thus,  numerous  studies  provide  evidence  that  the  presence  of  motion, 
i.e.,  movement  of  the  platform  upon  which  the  simulator  cockpit  rests,  does 
affect  performance  in  the  simulator.  Not  only  can  motion  affect  learning 
rates,  but  the  performance  of  the  pilot  in  the  presence  of  motion  may  be 
different  than  it  would  be  in  the  absence  of  motion.  With  motion,  his 
simulator  control  responses  to  external  forcing  functions  appear  to  be  more 
rapid  and  accurate  and  more  like  responses  used  to  control  the  aircraft 
in  flight.  While  it  cannot  be  concluded  from  these  studies  that  simulator 
motion  during  training  will  enhance  subsequent  performance  in  the  aircraft, 
they  do  suggest  that  simulator  motion  can  affect  the  acquisition  of  skills 
in  the  simulator.  These  effects  of  motion  upon  performance  in  the  simulator 
have  been  demonstrated  under  controlled  experimental  conditions  that  tend 
to  make  it  unlikely  that  the  noted  differences  in  performance  could  be 
attributed  solely  to  factors  other  than  the  presence  of  motion  during  simu- 
lator training. 

^Huddleston,  H.  F.  and  Rolfe,  J.  S.  Behavioral  Factors  Influencing 
the  Use  of  Flight  Simulators  for  Training.  Applied  F.rgonomics,  1971,  2.3, 
141-148. 
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Ince,  F.,  Wllliges,  R.  C.  and  Roscoe,  S.  N.  Aircraft  Simulator  Motion 
and  the  Order  of  Merit  of  Flight  Attitude  and  Steering  Guidance  Displays. 
Human  Factors,  1975,  17,  388-400. 
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The  influence  of  platform  motion  is  not  necessarily  beneficial,  however. 
Excessive  or  inappropriate  motion,  e.g.,  high  levels  of  simulated  turbulence, 
could  make  learning  less  rapid  if  it  were  a factor  in  making  the  simulator 
more  difficult  to  control.  Likewise,  motion  that  is  out  of  synchronization 
with  visual  or  other  cues  could  interfere  with  simulator  control  if  it  made 
trainees  ill  or  presented  misinformation  to  them.  For  example,  it  has  been 

reported  that  the  simulator  used  in  the  Air  Force  ASPT  study  cited  above 

% 

has  time  lags  in  the  motion  system  that  make  the  performance  of  some  maneu- 
vers difficult . ^ 

In  discussing  the  influence  of  motion  upon  pilot  performance  in  simu- 
2 3 

lators,  Gundry  ’ distinguishes  between  two  kinds  of  motion  cues  and  suggests 
that  they  might  affect  performance  differentially.  Maneuver  motion  is  that 
motion  that  arises  within  the  control  loop  and  results  from  a pilot-initiated 
change  in  the  motion  of  the  aircraft  in  order  to  change  its  heading,  alti- 
tude, or  attitude.  Disturbance  motion,  on  the  other  hand,  arises  outside 
the  control  loop  and  results  from  turbulence  or  from  failure  of  a component 

of  the  airframe,  equipment  or  engines  that  causes  an  unexpected  (to  tne 

4 

pilot)  motion  of  the  aircraft.  Matheny  made  a similar  distinction  in  a 
study  in  which  he  identifies  aircraft  motion  as  resulting  from  external 
forcing  functions  or  from  input  into  the  aircraft  controls. 

The  reason  that  platform  motion  can  result  in  quicker,  more  accurate 
simulator  control  probably  is  that  the  disturbance  component  of  that  motion 
resulting  from  simulated  turbulence  or  equipment  failure  can  provide  more 
rapid  and  relevant  alerting  cues  about  forces  acting  upon  the  aircraft  than 

^Hutton,  D.  P.,  Burke,  D.  K.,  Englehart,  J.  D.,  Wilson,  J.  M.  , Romaglia, 
F.  J.  and  Schneider,  A.  J.  Project  No.  2235,  Air-to-Ground  Visual  Simula- 
tion Demonstration  Final  Report,  Volume  1.  Aeronautical  Systems  Division, 
Simulator  SPO,  Wr ight-Patterson  AFB,  OH,  October  1976. 
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Gundry,  .1.  Man  and  Motion  Cues.  Th^’-’  Flight  Simulation  Symposium 
Proceedings . Royal  Aeronautical  Societx  . Lond  a,  April,  1976. 
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Inc.,  Hurst,  TX,  September,  1976. 
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could  be  obtained  from  other  cue  sources.  Maneuver  motion  does  not  fulfill 
an  alerting  function,  because  it  results  from  pilot-initiated  control  move- 
ments. Research  involving  maneuver  motion,  Gundry  states,  indicates  that 
this  component  of  platform  motion  has  little  effect  upon  the  control  of  an 
aircraft  whose  flight  dynamics  are  stable.  For  unstable  vehicles,  however, 
the  presence  of  maneuver  motion  will  allow  the  pilot  to  maintain  control 
even  in  flight  regions  where  control  by  visual  cues  alone  would  be  impossible. 
Thus,  disturbance  motion  permits  more  rapid  and  accurate  aircraft  control 
under  all  flight  conditions  in  which  such  motion  is  appropriate.  Maneuver 
motion,  however,  improves  aircraft  control  only  when  the  aircraft  is  un- 
stable . 

In  both  the  Jacobs  and  Roscoe  and  the  Air  Force  ASPT  studies  cited 
above,  emphasis  was  upon  simulation  of  maneuver  rather  than  disturbance 
motion.  Since  maneuver  motion  is  pilot  induced  and  the  aircraft  involved 
in  these  studies  were  quite  stable,  the  most  likely  role  of  potion  was  to 
provide  feedback  to  the  pilot.  If  sufficient  feedback  were  available  from 
other  sources  such  as  the  aircraft  instruments  or  an  extra-cockpit  visual 
display,  as  likely  was  the  case,  the  maneuver  motion  provided  in  these  two 
studies  could  not  be  expected  to  have  a large  effect  upon  simulator  training 
effectiveness,  and  probably  would  be  ignored  altogether  by  the  trainees. 

Had  these  two  studies  examined  the  influence  of  disturbance  moti.in  result- 
ing from  factors  outside  the  control  loop,  e.g.,  malfunctions,  the  results 
might  have  been  different. 

The  evidence  that  disturbance  motion  may  have  a large  effect  upon  pilot 
performance  in  the  simulator  and  upon  liis  subsequent  performance  in  the 
aircraft  should  not  be  overlooked  by  personnel  making  decisions  concerning 
the  Importance  of  platform  motion  in  aircraft  simulator  tra^"ing.  The 
fact  that  the  influence  of  such  motion  was  not  apparent  in  two  recent  trans- 
fer of  training  studies  is  probably  attributable  to  the  absence  of  a signifi- 
cant disturbance  component  to  the  motion  involved  in  those  studies.  The 
maneuver  motion  that  was  present  appears  not  to  have  been  a significant  fac- 
tor in  transfer  of  training  for  the  undergraduate  level  trainee  in  the  rela- 
tively stable  aircraft  involved  in  these  studies. 

The  majority  of  the  simulators  included  in  the  present  survey  had 
platform  motion  systems.  These  were  the  C-5,  the  FB-111,  the  F-4.  and  the 
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A-7.  Although  the  contribution  of  motion  to  the  effectiveness  of  the  CCT 
and  CT  activities  in  wliich  these  simulators  are  used  has  not  been  investi- 
gated empirically,  the  perceived  value  of  the  platform  motion  cues  they 
provide  was  discussed  with  the  Air  Force  pilots  and  instructors  interviewed. 
Many  of  these  personnel  held  strong  opinions  concerning  the  probable  value 
of  motion.  While  those  opinions  were  predominantly  favorable,  there  were 
a few  unfavorable  ones  as  well.  In  all  cases,  whether  favorable  or  unfavor- 
able, the  basis  for  tlie  opinions  expressed  w.is  explored  by  the  investigator. 

Favorable  opinions  were  difficult  to  relate  to  specific  aspects  of  motion 
simulation  and  in  many  cases  were  considered  to  be  endorsements  of  the  gen- 
eral idea  that  simulator  motion  is  important  because  the  aircraft  moves. 

Those  who  were  the  most  enthusiastic  in  their  opinions  favoring  motion  cited 
motion  characteristics  of  the  disturbance  type  as  the  primary  basis  for  their 
positive  views,  i.e.,  motions  associated  with  equipment  failure,  weapons  re- 
lease, buffet  and  turbulence.  Very  few  pilots  expressed  strong  positive 
feelings  toward  maneuver  motion — it  contributed  to  realism,  but  was  not  cited 
as  specifically  related  to  particular  training  goals. 

The  relatively  few  negative  opinions  expressed  concerning  motion  were 
ail  strongly  held,  and  they  involved  both  maneuver  and  disturbance  motion. 

In  one  simulator  in  which  maneuver  motion  cues  lagged  noticeably  behind 
instrument  displays  and  tended  to  be  jerky  rather  than  smooth,  the  motion  was 
viewed  as  annoying,  and  pilots  using  that  particular  device  preferred  to 
train  with  the  motion  system  inoperative.  Apparently,  maneuver  motion  can 
have  little  positive  value  in  most  simulator  training  programs,  but  it  can 
have  a negative  influence,  at  least  upon  pilot  attitudes,  if  it  is  not  rep- 
resentative of  comparable  motion  in  the  aircraft,  e.g.,  if  it  lags  notice- 
ably the  pilot's  control  input  or  the  cues  provided  by  instruments  or  visual 
displays. 

Most  of  the  negative  comments  received  during  the  interviews  could  be 
related  to  disturbance  motion — or  more  precisely,  to  the  absence  of  dis- 
turbance motion  cues  that  the  pilots  knew  to  be  characteristic  of  the  air- 
craft simulated.  Two  examples  of  this  situation  were  noted,  one  involving 
the  A-7D  simulator  and  the  other  involving  the  C-5A  simulator. 

A critical  condition  involving  the  A-7D  aircraft  is  that  labeled 
"Departure."  It  is  a high  angle  of  attack  condition  in  which  the  aircraft 
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yaws  abruptly  and  enters  an  uncontrollable  spin.  The  yaw  in  this  case  is 
a disturbance  cue  that  alerts  the  pilot  to  the  condition's  onset.  Training 
in  recovery  from  the  departure  is  considered  of  critical  importance,  but 
such  training  in  tlie  single-place  aircraft  is  not  included  in  A-7D  training 
for  reasons  of  fliglu  safety.  Attempts  to  provide  the  desired  training  in 
the  A-7D  simulator  at  Davi s-Monthan  Air  Force  Base  have  been  unsuccessful, 
reportedly  because  the  yaw  motion  cues  cannot  be  simulated  in  that  device 
(it  lacks  the  yaw  motion  axis). 

The  A-7D  manufacturer  has  a research  simulator,  called  LAJiBS  (for 
Large  Amplitude  Motion  Base  Simulator),  in  which  the  departure  can  be  simu- 
lated with  the  yaw  motion  component.  All  Air  Force  A-7D  pilots  have  under- 
gone departure  recovery  training  in  LAMBS.  Pilots  who  have  previously  ex- 
perienced a departure  in  the  aircraft  have  reported  that  its  simulation  in 
LAMBS  is  "realistic."  Pilots  who  received  training  in  LAMBS  and  subsequently 
experienced  a departure  in  the  aircraft  credit  the  motion  simulator  train- 
ing for  the  ease  with  which  they  were  able  to  reestablish  aircraft  control, 
although  such  reports  are  purely  subjective.  In  any  event,  the  disturbance 
cue  provided  by  LAMBS  is  well  re<’eived  by  Air  Force  A-7  pilots,  whereas  the 
absence  of  that  cue  in  the  Air  Force  simulator  renders  departure  recovery 
training  in  it  unacceptable. 

One  of  the  A-7D  pilots  interviewed  had  recently  flown  the  Navy's  A-7E 
simulator  which  has  a six-axis  motion  system.  That  pilot  expressed  a high- 
ly favorable  opinion  concerning  the  worth  of  A-7E  simulator  training  and  a 
clearly  unfavorable  opinion  concerning  A-7D  simulator  training.  While  there 
are  a number  of  differences  in  these  two  simulators  and  the  manner  in  which 
they  are  used,  the  extent  of  motion  simulation  was  singled  out  as  an  im- 
portant difference  between  the  two  devices  which,  in  the  opinion  of  that 
particular  pilot,  influences  training. 

The  C-5A  simulator  motion  platform,  like  that  of  the  A-7D,  lacks  the 
yaw  axis,  so  the  yaw  disturbance  motion  associated  with  loss  of  an  engine 
cannot  be  simulated.  Pilots  associated  with  C-5A  training  at  Aitus  AFB  cited 
this  deficiency  in  the  simulator  as  a negative  factor  in  determining  the 
effectiveness  of  simulator  training  for  engine  losses  at  low  altitude.  It 
was  not  cited  as  a factor  in  other  training  operations,  including  operations 
that  involve  yaw  maneuver  motion,  however.  Thus,  where  yaw  could  he  con- 
sidered a disturbance  motion,  it  was  perceived  to  be  needed  for  training; 


i; 

f 


where  it  could  be  considered  a maneuver  motion,  it  was  not  perceived  as 
important . 

The  C-5  simulator  at  Altus  has  a visual  display,  and  the  yaw  of  the 
aircraft  associated  with  engine  loss  is  reflected  in  the  visual  scene.  The 
pilots  indicated  that  the  visual  yaw  cue  alone  was  insufficient,  in  their 
i opinion,  when  engine  loss  occurred  during  landing  and  takeoff  maneuvers. 

They  felt  that  pilots  in  the  simulator  responding  to  visual  cues  were  much 
slower  in  initiating  corrective  action  than  they  were  in  the  aircraft  where 
motion  provides  an  early  and  more  pronounced  alert  that  a disturbance  has 
occurred . 

The  influence  of  platform  morion  upon  transfer  of  simulator  training 
has  not  been  clearly  established  by  the  liata  available  at  the  present  time. 
It  has  been  demonstrated  that  motion  can  affect  pilot  performance  in  the 
simulator  in  ways  that  may  make  his  performance  in  the  simulator  more  like 
his  performance  in  the  aircraft,  but  it  lias  not  been  shown  that  simulator 
motion  enhances  his  subsequent  performance  in  the  aircraft.  The  two  studies 
that  have  addressed  Che  question  of  transfer  directly  did  not  support  a 
conclusion  that  motion  is  needed.  Likewise,  there  is  no  consensus  among 
pilots  as  to  the  need  for  motion  in  simulator  training. 

The  distinction  made  by  Gundry  between  maneuver  and  disturbance  motion 
is  useful  in  attempting  to  understand  both  the  prior  research  on  motion  and 
the  reactions  of  pilots  to  the  motion  component  of  aircraft  simulators.  It 
appears  that  more  attention  has  been  paid  in  the  design  of  current  simula- 
tors to  maneuver  motion  than  to  disturbance  motion.  Kmphasis  has  been  upon 
providing  in  a simulator  the  motion  cues  associated  with  well-coordinated 
pilot  control  inputs,  scaled  down  to  the  limits  of  travel  and  accelerations 
of  the  motion  platform.  Since  most  training  and  operational  aircraft  are 
relatively  stable,  this  kind  of  motion  simulation  may  be  of  very  little  po- 
tential value  in  training.  It  might  be  more  beneficial  from  the  training 
standpoint  to  provide  the  motion  cues  associated  with  disturbances  to  the 
aircraft  not  originated  by  the  pilot,  and  then  only  at  their  initial  onset 
values,  so  that  he  could  learn  to  respond  specifically  to  such  cues  rather 
than  learning  to  respond  to  visual  or  other  cues  that  occur  later  in  time. 

The  area  of  motion  simulation  is  in  need  of  further  investigation. 
Future  stud  es  of  motion  should  concentrate  upon  specific  motion  rues  of 
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a disturbance  nature  that  can  be  related  to  specific  training  objectives. 

If  motion  is  to  have  an  influence  upon  simulator  training  effectiveness,  it 
would  appear  that  such  an  influence  will  be  found  in  critical  flight  maneu- 
vers, not  in  routine  maneuvers  performed  under  optimum  flight  conditions. 

Handling  Characteristics 

None  of  the  simulators  included  in  the  present  survey  was  new,  and  most 
were  designed  from  one  to  three  decades  ago  and  employ  analog  computational 
technology.  Consequently,  the  extent  to  which  the  handling  characteristics 
of  those  simulators  approximate  the  characteristics  of  the  aircraft  being 
simulated  varies  widely,  with  the  older,  analog  devices  being  considered 
less  like  the  aircraft  than  the  newer  devices.  Many  of  the  pilots  inter- 
viewed commented  unfavorably  upon  the  handling  characteristics  of  the  simu- 
lators in  which  they  were  training  and  expressed  views  that  related  the 
"unrealistic"  handling  characteristics  to  limitations  upon  the  device's 
training  effectiveness.  In  fact,  the  general  view  was  that  a device  that 
did  not  realistically  represent  the  aircraft  was  useful  only  as  a proce- 
dures trainer  in  which  non-flying  tasks  could  be  practiced. 

The  literature  on  tr..nsfer  of  training  does  not  fully  support  these 
pilot  opinions,  although  supporting  views  have  been  expressed  by  an  earlier 
reviewer.^  When  the  simulation  algorithm  grossly  distorts  the  character 
of  the  simulation,  as,  for  example,  when  the  simulator  includes  an  unreal- 
istic control  gain,  a small  negative  effect  upon  transfer  of  training  has 
2 

been  demonstrated.  On  the  other  hand,  two  studies  in  which  the  corres- 
pondence in  control  pressures  between  the  simulator  and  the  aircraft  was 
varied  concluded  that  transfer  was  ".ot  affected  by  large  differences  in 

^Gerathewohl , S.  .1.  Fidelity  of  Simulation  and  Transfer  of  Training: 

A Review  of  the  Problem.  Report  No.  AM  69-24,  Federal  Aviation  Administra- 
tion Office  of  Aviation  Medicine,  Washington,  DC,  December  1969. 

2 

Muckier,  F.  A.,  Obermeyer,  W.  H.,  Hanlon,  W.  H.  and  Serio,  F.  P. 
Transfer  of  Training  With  Simulated  Aircraft  Dynamics:  IT.  Variations 

in  Control  Gain  and  Physical  Characteristics.  WADD  Technical  Report  60- 
615  (II),  Aerospace  Medical  Researcli  Laboratories,  Wr ight-Patterson  AFB, 

OH,  December  1961. 
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1 2 

control  pressure.  ’ Instead,  transfer  depended  more  upon  correspondence 

between  the  pattern  of  control  movements  required  in  the  simulator  and ' in 

the  aircraft  than  upon  the  amount  of  control  force  required.  Although 

varving  the  control  pressures  affected  the  case  with  which  trainees  learned 

to  control  a device,  its  transfer  of  training  value  was  unaffecti-d.  Udiere 

the  correspondence  between  the  device  and  the  aircraft  is  gross,  however, 

as  was  reported  for  one  device  in  which  forward  pressure  on  the  wheel 

3 

resulted  in  a climb  configuration,  simulator  effectiveness  undoubtedly 
will  suf f er . 

Although,  in  the  extreme  case,  simulator  response  characteristics 
unlike  those  of  the  aircraft  can  produce  negative  transfer  of  training, 
there  is  little  evidence  that  the  simulator  must  precisely  duplicate  the 
feel  of  the  aircraft  in  order  to  be  effective.  It  is  possible,  however, 
that  even  minor  dissimilarities  in  feel  or  response  could  lead  to  the 
same  kinds  of  potential  problems  found  in  simulators  without  motion,  i.e., 
lower  correlation  between  simulator  and  flight  performance,  particularly 
where  the  more  difficult  to  fly  aircraft  are  c(!ncerned.  The  more  likely 
influence  of  poor  correspondence  between  the  simulator  and  the  aircraft 
is  upon  the  attitude  of  the  personnel  involved.  Pilots  have  resisted, 
and  most  likely  will  continue  to  resist,  using  a simulator  that  does  not 
"feel"  like  the  aircraft.  In  fact,  aircraft  "feel"  has  become  a principal 
criterion  for  simulator  design.'* 

^Matheny,  W.  G. , Williams,  A.  C. , Ir.,  Dougherty,  D.  .1.  and  Hasler, 

S.  G.  Thj?  Effect  of  Varying  Control  Forces  in  the  P-1  Trainer  Upon  Transfer 
of  Training  to  the  T-b  Aircraft.  Technical  Report  HRRC-TR-53-31 , Human 
Resources  Research  Center.  Goodfellow  AFB,  TX,  1953. 

"“Wilcoxon,  H.  C.  and  Daw,  E.  Fideli  ty  of  Simulation  in  Operational 
Flight  Trainers,  Part  II:  _|he  Effect  of  Variations  in  Control  Eoadings  oji 
the  Tra  m inj^^  Va  lue^  o f the  SN-f  CIFTE  Techni  cal  Report  SPECDEVCEN999-2-3bV 
Special  Device  Center,  Port  Washington,  N5’ , January  1954. 

3 

Military  Potential  Test  of  Fixed  Wing  Basic  Instrument  Trainer  2-B- 1 2A. 
U.  S.  Army  Aviation  Test  Board  Report  of  Test  I'SATECOM  Project  No.  4-3-5150- 
01-9,  C . S.  Army  Aviation  Test  Board,  Fort  Rucker,  Ah,  1963. 

4 

Catron,  R.  h.  A New  Approach  for  Establishing  Aerodynamics  Perfor- 
mance of  Flight  Trainers.  Ht_h  NTEC/ 1 ndustry  Conference  Proceedings.  Naval 
Training  Equipment  Center.  Orlando,  FT,  November  1975. 
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A General  Conclusion  Concerning  Design 


Understanding  of  the  relation  of  simulator  design  features  discussed 
above  to  simulator  training  effectiveness  is  quite  limited.  It  is  clear 
that  designing  a simulator  to  be  used  for  training  is  not  entirely  a 
matter  of  duplicating  an  aircraft.  Instead,  it  appears  to  be  a matter  of 
providing  a learning  environment  in  which  precisely  specified  training 
objectives  may  be  addressed.  If  a particular  feature  of  the  aircraft  and 
its  physical  environment  is  necessary  to  the  performance  of  a particular 
task  in  flight,  rhea  that  feature,  wliether  it  is  a dimension  of  motion,  an 
ECM  panel,  or  a peculiar  response  characteristic,  must  be  represented  in 
some  fashion  in  the  simulator  if  the  device  is  to  be  used  to  practice  or 
learn  such  a task.  If  the  task  can  be  performed  or  approximated  in  a 
simulator  that  lacks  a particular  feature  of  the  aircraft,  then  that 
feature  may  have  little  or  no  influence  upon  simulator  effectiveness  for 
the  learning  of  that  task.  The  physical  correspondence  between  the  simu- 
lator and  the  aircraft  is  probably  more  related  to  cost,  as  Miller^  indi- 
cated almost  two  decades  ago,  than  to  training  effectiveness.  If  the 
degree  of  correspondence  between  the  device  and  the  aircraft  is  relevant 
to  the  objectives  of  the  intended  training,  training  in  the  simulator  can 
be  made  effective;  whether  it  is  or  not  is  a matter  related  to  other 
factors . 

Training  Programs 

Frequent  note  has  been  taken  of  the  influence  upon  training  effective- 
ness of  the  manner  in  which  a simulator  is  used.  Yet,  the  literature  is 

full  of  reports  of  situations  in  which  the  Importance  of  training  program 

2 3 

design  and  execution  seemed  to  be  ignored.  ’ Although  there  is  an 

^Miller,  R.  B.  Psychological  Considerations  in  the  Design  of  Train- 
ing F.quipment,  Technical  Report  WADC  TR-54-563,  Wright  Air  Development 
Center,  Wrlght-Patterson  AFB,  OH,  December  1974. 

2 

Hall,  E.  R. , Parker,  ,1.  F. , dr.  and  Meyer,  D.  E.  A Study  of  Air  Force 
Flight  Simulator  Programs.  Technical  Report  AMRL-TR-67- 1 1 1 , Aerospace 
Medical  Research  Laboratories,  Wr i ght-Patterson  AFB,  OH,  1967. 

3 

Casperson,  R.  C.  and  Channell,  R.  C.  Use  of  the  Operational  Flight 
Trainer . Technical  Report  NAVTRADEVCEN  1734-00-1,  Naval  Training  Device 
Center,  Port  Washington,  NY,  May  1957. 


increasing  emphasis  upon  effecfive  use  of  device,  instances  continue  to-,b<» 
encountered  in  which  simulators  are  misusc-d  or  are  used  inef f i ci ent 1 v hy 
almost  all  military  and  civilian  pilot  training  agencies.  Even  the  current 
research  literature  contains  repc>rts  of  simulator  effectiveness  stuiiies  in 
which  participating  instructors  were  permitted  to  conduct  training  in  non- 
standardized  ways. 

To  list  all  training  program  design  and  execution  variables  that 
potentially  influence  simulator  training  effectiveness  would  be  an  almost 
interminable  task.  Any  of  the  numerous  textbooks  on  human  learning  will 
provide  a source  for  identification  of  variables  that  influence  learning 
and  performance,  e.g. , schedules  of  reinforcement,  meaningful  ness  and  diffi- 
culty of  material  to  be  learned,  size  of  learning  blocks,  anticipatory 
set,  and  knowledge  of  results.  Flexman,  Matheny,  and  Rrown^  have  shown 
how  such  variables  can  be  employed  to  increase  simulator  and  flight 
training  effectiveness. 

During  the  current  survey  of  Air  Force  simulator  training  activities, 
a number  of  training  program  features  were  noted  that  might  be  suspected 
to  affect  simulator  training  effectiveness.  For  example,  it  is  a common 
practice  in  Air  Force  CCT  programs  to  alt('rnate  i nst ructional  periods  be- 
tween the  simulator  and  the  aircraft.  Wiile  the  specific  pattern  of  al- 
ternation varies  from  program  to  program,  a tvpical  pattern  would  involve 
a period  of  simulator  training  only,  followed  hy  a periiui  during  which 
consecutive  missions  might  alternate  between  the  simulator  and  the  air- 
craft, and  finally,  a period  during  which  all  training  missions  are  con- 
ducted in  the  aircraft.  The  emphasis  upon  earlv  training  in  the  simulator 
and  later  training  in  the  aircraft  seems  reasonable  and  probably  Is 
dictated  in  part  by  the  availability  of  both  simulators  and  aircraft,  as 
well  as  the  "realism"  of  the  simulators  themselves.  Of  interest  in  surh 
programs,  however,  is  the  question  whether  the  alternation  between  two 

'plexman,  R.  F.,  Matbenv,  W.  G.  and  brown,  F.  G.  Evaluation  of  the 
School  Link  and  Special  Methcnls  of  rnstruction  in  _2.  Jf'lt'j-hjVij'  Pri  vate  Pi  lot 
Flight-Training  Program.  University  of  Illinois  Aeronautics  Rulletin  No. 

8,  Urbana,  IT,  19S0. 
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training  vehicles,  particularly  in  those  cases  in  which  the  simulator  might 
be  judged  "unrealistic,"  might  influence  the  effectiveness  of  simulator 
training. 

The  sequencing  of  simulator  and  aircraft  training  has  been  suggested 
elsewhere  as  a factor  that  could  influence  the  effectiveness  of  simulator 
training.  Smode , Hall  and  Meyer'  concluded  that  the  evidence  concern- 
ing the  influence  of  sequencing  was  i nconc lus i ve , although  they  sug- 
gested there  might  be  circumstances  in  which  a sequence  that  alternates 

between  simulator  and  aircraft  would  be  beneficial.  Meister,  Sullivan, 

2 

Thompson  and  Finley  presented  data  that  suggest  that  switching  from  the 
aircraft  to  the  simulator  reduced  performance  in  the  simulator  on  the 
following  sessions,  resulting  in  a training  inefficiency.  From  a purely 
economic  standpoint,  it  would  appear  desirable  to  train  first  in  the  less 
expensive  (presumably)  simulator  and  transfer  that  training  or  resultant 
performance  to  the  aircraft,  lihlle  there  may  be  some  interactive  effects 
between  the  sequence,  the  manner  in  whlcli  the  device  is  used,  and  the 
design  of  the  device  that  could  influence  effectiveness,  it  would  appear 
quite  likely  that  training  in  the  aircraft  before  the  full  benefit  of  the 
simulator  has  been  realized  with  respect  to  a particular  task  would  tend 
to  reduce  the  overall  efficiency  of  the  simulator-aircraft  training  pro- 
gram. In  an  unpublished,  non-Air  Force  effort  that  illustrates  this  view, 
a fifty  training  hour  program  in  which  the  simulator  was  used  prior  to 
training  in  the  aircraft  became  a sixty  training  hour  program  when  the 
sequence  was  changed  to  mix  simulator  and  aircraft  training,  although 

' Smode,  A.  F. , Hall,  E.  R.  and  Meyer,  D.E.  An  Assessment  of  Research 
Relevant  to  Pilot  Training.  Technical  Report  AMRL-TR-66- 1 96 , Aerospace 
Medical  Research  Laboratories,  Wright-Patterson  AFR,  OH,  1967. 

2 

Meister,  D. , Sullivan,  D.  J.,  Thompson,  E.  A.  and  Finley,  D.  L. 

Train ing  Effectiveness  Evaluation  of  Naval  Train! ng  Devices  Part  II.  jA 
Study  of  Device  2F66A  (S-2E  Trainer)  Effectiveness.  Technical  Report 
NAATRADEVCEN  69-C-0332-2,  Naval  Training  Device  Center,  Orlando,  FL, 

1971. 
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other  procedural  changes  were  introduced  concurrently  that  could  have 
contributed  to  the  resulting  inefficiency. 

Training  program  content  is  an  obvious  influence  upon  simulator 
training  effectiveness.  A dynamic  flight  simulator  used  only  as  a proce- 
dures trainer,  for  example,  is  not  being  used  effectively,  and  transfer 
of  training  is  not  good  among  students  who  must  learn  to  do  one  thing  in 
a simulator  and  something  else  in  an  aircraft,  an  unintended  consequence 
that  could  possibly  result  from  continued  use  of  obsolete  simulators  such 
as  the  F-4  and  B-52  simulators  examined  during  the  current  survey. 

Other  potential  problems  related  to  the  content  of  simulator  train- 
ing were  noted  during  the  survey.  In  several  CCT  and  CT  programs,  for 
example,  non-usable  information  included  in  the  training  appeared  to  be 
having  a negative  influence  upon  simulator  training  effectiveness  because 
of  time  being  devoted  to  it  rather  than  to  other,  more  important  infor- 
mation. Often,  in  the  course  of  a training  mission  in  one  of  these  pro- 
grams, it  was  reported  that  students  ask  questions  about  the  aircraft  that, 
while  of  passing  interest,  are  not  relevant  to  the  training  objectives 
being  addressed.  If  the  Instructor  could  answer  the  question  (Instructors 
typically  take  pride  in  being  able  to  supply  upon  demand  the  most  obscure 
and,  from  the  cockpit  at  least,  unusable  bits  of  Information  about  the 
aircraft),  he  would  do  so,  thus  possibly  reducing  training  effectiveness 
by  overloading  the  student  with  information  that  he  would  not  be  able  to 
use  while  performing  as  an  aircrew  member  and  additionally  reducing  the 
training  time  available  to  provide  more  useful  information.  Presumably, 
as  on-going  Air  Force  ISO  efforts  are  completed,  the  amount  of  non-usable 
information  in  such  programs  will  be  reduced. 

Instances  were  also  noted  in  which  the  subject  matter  addressed  in  a 
simulator  training  program  did  not  appear  appropriate  to  the  features  of 
the  simulator.  In  one  such  instance,  it  was  reported  that  the  questionable 
subject  matter  was  added  at  the  direction  of  a commander  not  familiar  with 
the  simulator  in  question.  The  consequence  was  that  the  effectiveness  of 
simulator  training  was  reduced,  at  least  for  the  time  during  which  the 
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inappropriate  activity  was  in  progress.  In  such  a situation,  the  ineffec- 
tive training  program  content  might  be  attributed  more  appropriately  to  the 
simulator  training  personnel  who  rigidly  carried  out  the  instruction  with- 
out attempting  to  inform  the  commander  of  the  limitations  of  the  device, 
than  to  the  commander  wlio  attempted  to  improve  the  performance  of  his 
pilots  but  was  not  given  adequate  feedback  concerning  his  attempts  to 
achieve  that  purpose. 

In  spite  of  the  potential  training  advantages  simulators  offer  over 
aircraft,  there  is  a tendency  in  some  simulator  training  programs  to 
ignore  those  advantages  and  treat  simulators  as  though  they  were  aircraft. 
While  this  practice  reflects  a favorable  attitude  toward  simulators,  it 
may  sometimes  reduce  simulator  effectiveness  simply  because  it  preserves 
some  of  the  disadvantages  of  the  aircraft  as  a training  vehicle.  The 
scheduling  of  long  training  periods  in  the  simulator  because  missions  in 
the  aircraft  being  simulated  are  necessarily  long  is  an  example  of  such 
a practice.  Another  example,  reported  by  personnel  at  one  base  visited, 
was  the  requirement  that  all  debriefing  activity  be  deferred  until  after 
the  mission  rather  than  to  lose  valuable  simulator  time  telling  students 
what  they  may  have  done  right  or  wrong  in  the  simulator.  While  such  a 
practice  may  have  merit  in  several  respects,  it  fails  to  take  full  ad- 
vantage of  the  ease  with  which  a simulated  mission  can  be  interrupted  in 
order  to  give  trainees  feedback  concerning  performance  and  the  benefits 
in  terms  of  learning  efficiency  such  a procedure  would  present. 

The  duration  of  a training  period  in  the  simulator  was  cited  by  a 
number  of  the  personnel  interviewed  as  a significant  factor  in  simulator 
training  effectiveness.  To  a degree,  their  judgments  as  to  optimum 
period  length  was  a function  of  the  type  of  aircraf t/misslon  being  simu- 
lated. Pilots  of  the  small-crewed  fighter  aircraft  that  engage  in  high 
intensity  missions  prefer  shorter  periods  than  do  those  of  larger-crewed 
transports  and  bombers  who  typically  engage  in  less  intensive  missions. 
Where  simulator  training  consisted  of  high  intensity  activities  such  as 
frequent  systems  failure,  regardle.ss  of  aircraft  type  or  other  considera- 
tions, pilots  suggested  that  training  periods  of  from  90  to  120  minutes 
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duration  are,  in  their  judjtment,  of  more  value  than  either  shorter  or 
longer  periods.  The  tendency  to  judge  the  validity  of  simulator  training 
program  period  length  hy  comparing  it  to  aircraft  missions  would  appear 
to  have  little  merit  and  to  load  to  reduced  training  efficiency  where  the 
period  is  of  such  duration  that  fatigue  can  become  a factor  affecting 
simulator  training  effectiveness.  Unfortunately,  there  is  no  directly 
applicable  research  data  that  can  be  used  as  a basis  for  specifying  the 
optimum  duration  of  a simulator  training  mission. 

The  schedule  of  training  activities,  that  often  was  designed  to 
optimize  use  of  training  resources,  could  also  be  a factor  affecting  simu- 
lator trairing  effectiveness  under  some  c i rcumstances . For  example,  one 
instance  was  cited  by  a group  of  trainees  in  which  a 4h-hour  simulator 
training  mission  was  completed  at  midnight,  and  its  post-mission  debriefing 
was  completed  at  3:00  a.m.  The  effectiveness  of  such  a long  training 
mission  might  be  questioned  purely  on  the  basis  of  probable  trainee  fa- 
tigue, as  could  the  contiguous  three-hour  debriefing,  althovigh,  as  indi- 
cated above,  there  is  no  directiv  relevant  data  concerning  optimum  length 
of  such  training  activities.  Additionally,  the  efficiency  of  training  at 
such  hours  of  the  morning  might  be  questioned  on  the  basis  of  available 
information  concerning  diurnal  rhythms.  To  compound  the  possible  loss  of 
effectiveness  of  such  a training  schedule,  these  same  students  reportedly 
were  scheduled  for  another  simulator  training  mission  at  9:00  a.m.  It 
is  understood  that  such  tight  scheduling  of  simulator  training  is  not 
common,  however. 

flvjre  are  a number  of  other  training  program  factors  that  influence 
the  efficiency  of  simulator  training,  although  not  necessarily  its  effec- 
tiveness. For  example,  it  has  also  been  reported  that  simulator  train- 
ing presented  in  the  context  of  simulated  mission  activities,  as  opposed 
to  abstract  training  exercises,  tends  to  be  more  effective,^  and  the 

^Caro,  P.  W.  An  innovative  Instrument  Flight  Training  Program. 

Paper  No.  710480,  .Society  of  Automotive  Fngineers,  New  York,  1971. 
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liternture  on  learning  ami  forgetting  suggests  that  beliavior  learned 
within  such  a meaningful  context  will  be  less  quickly  forgotten.*  Several 
anecdotes  were  cited  by  aircrews  interviewed  that  support  these  researi  li 
findings.  Other  factors  include  the  amount  of  simulator  training.,  t lie 
sequence  in  which  instruction  is  conducted  in  the  simulator,  the  use  of 
individual  (as  opposed  to  group)  pacing,  training  to  specified  criteriim 
levels  (as  opposed  to  trainin.;  for  fixed  time  periods),  and  the  i-xtent  ti' 
which  simulator  training  includes  tasks  that  can  be  U-arned  more  effi- 
ciently  in  the  aircr.ift.  Smode,  Hall  and  Meyer'  pointi'd  out  a decade  ago 
that  little  was  known  ab<nit  how  to  manipulate  stub  fac  tors  to  best  advan- 
tage. That  observation  is  still  v.alid. 

Personae  1 

Simulator  training  involves  trainees  and  instrcictors.  Roth  cate- 
gories of  personnel  re[>resent  potential  influences  uptui  I'f  f ect  i veness  . 

The  most  obviously  relevant  considerations  with  respect  to  both  are  their 
qualifications  and  prior  experience,  but  occasionally  other  variables  are 
suggested.  For  example,  .-leister,  Sullivan,  Thompson,  and  Finlev*  found  a 
difference  in  the  effectiveness  of  one  simulator  training  program  for 
student  and  operational  pilots  vs.  reservists.  The  ifference  could  also 
be  attributed  to  considerations  such  as  fatigue  and  stress,  factors  which 
probably  account  for  many  unexpected  findings  in  learning  studies.  ITie 
present  report  will  discuss  only  the  more  obvious  persc'nnel  factors. 

*.Ienkins,  d.  d.  Remember  That  Old  Theory  of  Memory?  Well  Forget  It! 
Arne rlcan  Psychologis t , 1 H 7 4 , 79,  7 8 b- 7 9 1 . 

2 

Smode,  A.  F.  , Hall,  E.  R.  and  Meyer,  11.  K.  An  Assessment  c^f  Research 
Relevant  to  Pilot  Training.  Technical  Report  AMKl.-TR-bb- 1 9b , Aerospaee 
Medical  Research  Laboratories,  Wright-Pat terson  .\FR,  OH,  19p7. 

3 

Mcister,  t).  , Sullivan,  I).  .1.,  ITiompson,  E.  A.,  .and  Finlev,  11.  I.. 
Training  Effectiveness  Evaluation  of  Nav.aJ^  J’ra  i n i ng  Devices  Part  II-  A 
Study  of  Device  2FfibA  (S-2E  Trainer)  Ef feet i vt-ness . Technical  Report 
NAVTRADEVCEN  69-C-0332-2,  Naval  Training  Devi'ce  Center,  Orlando,  FI.,  1971. 


bO 


Irainees.  All  investigations  of  human  learning  are  subject  to  tlie  influ- 
ences of  task  relatcil  aptitudes  of  the  learners.  Aptitudes  are  defined  in 
terms  of  learning  efficieii'  y,  and  high  aptitude  students  learn  a given  task 
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mort'  rapidlv  or  to  a greater  degree  than  tin  low  aptitude  students.  Where 
the  training  program  involvi-s  fixed  ami’unts  of  simulator  training  time, 
high  aptitude  student'.  K'.irn  more  tasks  to  transfer  to  the  aircraft;  where 
training  i to  lixed  perfiirm.ince  levels  and  training  time  varies,  high 
.and  low  .iptitude  students  .achieve  .about  equally,  but  high  aptitude  students 
require  les.s  training  time  in  the  simulator.  A measure  of  simulator  train- 
ing efficiency  such  as  the  Transfer  Kf f ec t i veness  Ratio^  will  yield  a higher 
value  for  high  aptitude  students,  but  this  does  not  indic.ate  that  the  simu- 
lator training  program  is  more  effective  with  such  students.  It  is  probably 
equally  effective  with  both  groups  of  students,  but  training  time  in  the 
device  will  be  shorter  for  one  than  for  the  other.  Thus,  while  high  aptitude 
students  learn  more  efficiently,  .aptitude  per  se  is  not  believed  to  be  an 
influence  upon  simulator  training  effectiveness. 

The  influence  upon  .aimujalor  training  effectiveness  of  level  of 
tr.iinee  skill  or  amount  of  prior  flight  experience  is  frequently  ques- 
tioned. Many  military  pilots  .and  managers,  including  some  of  those  inter- 
viewed during  the  present  survey,  acknowledge  that  simulators  provide 
appropriate  training  for  the  airlines,  where  the  trainees  are  highly 
experienced,  but  insist  that  the  devices  cannot  be  relied  upon  as  ex- 
tensively to  trtiin  less  experienced  military  pilots,  particularly  those 
with  fewer  th;in  about  1,000  hours  flight  time.  The  skills  possessed  by 
thesi.'  two  groups  of  tr.ainees  do  differ,  qualitatively  as  well  as  quanti- 
tativelv,  and  the  tasks  for  which  they  undergo  training  often  are  not 
identic.al.  Iherefore,  the  training  they  receive  should  not  necessarily 
he  identical  if  it  is  optimally  designed  to  meet  their  respective  training 

^ I’ovemnire , II.  K.  , and  Roscoo , S.  N.  An  Evaluation  of  Cround-B.ased 
flight  Trainers  in  Routine  I’rimarv  Kliglit  Training.  Human  Factorj^,  1971, 

1 1,  109-1  lb. 
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needs,  and  the  characteristics  of  the  simulators  involved  in  their  traininR 

might  vary  as  well.  it  does  not  follow,  however,  that  simulator  training 

can  he  appropriately  designed  and  conducted  for  one  experience  level  trainee 

but  not  for  another.  In  fact,  the  experimental  evidence  does  not  support 

the  cc.-t-ntlon  that  simulator  training  effectiveness  is  influenced  by  level 

of  trainee  experience  in  isolation  from  other  factors.  After  reviewing 

a large  number  of  transfer  of  training  studies,  Micheli^  concluded  that 

flight  training  devices  are  effective  for  both  neophyte  pilot  trainees 

2 

and  airline  pilots.  Further,  Brictson  and  Burger  found  that  simulator 
training  administered  to  both  low-  and  high-time  pilots  was  effective 
for  both  groups,  although  it  reduced  the  differences  in  performance  be- 
tween the  two  groups  during  subsequent  carrier  landing  training  and  in 
their  respective  attrition  rates. 

One  trainee-related  consideration  that  possibly  Influences  simulator 
training  effectiveness  is  proficiency  at  the  time  of  simulator  training. 

A Continuation  Training  program,  particularly  one  not  based  upon  pro- 
ficiency as  was  the  case  with  all  of  the  CT  programs  included  in  the  cur- 
rent survey,  will  be  of  less  value  for  pilots  who  fly  operational  missions 
daily  than  for  pilots  assigned  to  staff  positions  whose  flying  opportuni- 
ties are  limited.  A given  training  program  can  be  effective  for  a pilot 
whose  proficiency  is  low,  but  ineffective  for  a proficient  pilot  because 
of  the  performance  proficiencies  and  deficiencies  associated  with  each. 

Crew  position  is  another  factor  that  probably  influences  the  effec- 
tiveness of  some  of  the  simulator  training  programs  Included  in  the 

^Mlcheli,  G.  Analysis  of  the  Transfer  of  Training,  Substitution  and 
Fidelity  of  Simulation  of  Training  Equipment.  TAEG  Report  2,  Naval 
Training  Equipment  Center,  Orlando,  FL,  1972. 

2 

Brictson,  C.  A.  and  Burger,  W.  .1.  Transfer  of  Training  Effective- 
ness . A-7E  Night  Carrier  Landing  Trainer  (NCLT)  Device  2F103.  Technical 
Report  NAVTRAEQUIPCEN  74-C-0079-1,  Naval  Training  Equipment  Center, 

Orlando,  FL,  August  1976. 
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current  survey.  The  Weapons  Systems  Officer  in  the  F-4E  Continuation 
Training  program  at  Eglin,  for  example,  appears  to  derive  more  training 
benefits  from  the  F-4E  simulator  than  does  the  pilot.  Two  factors  that 
appear  to  influence  tliis  apparent  difference  are:  (1)  the  pilots  are 

reportedly  proficient  in  most  of  the  syllabus  missions  before  engaging  in 
simulator  training  by  virtue  of  having  flown  similar  missions  in  the  air- 
craft; and  (2)  the  WSOs  are  able  to  practice  syllabus  tasks  in  the  simu- 
lator that  they  cannot  practice  in  the  aircraft.  These  tasks  include 
operation  of  EW  equipment  that  cannot  be  operated  in  the  aircraft  for 
security  reasons  and  performing  intercepts  to  develop  levels  of  proficiency 
not  attainable  during  their  limited  training  and  operational  flights  in 
the  aircraft. 

Instructors.  After  reviewing  the  literature  on  the  flight  instructor,  Smode , 
Hall  and  Meyer^  concluded  that  there  is  no  empirical  basis  for  the  notion  that 
experienced  pilots  make  better  in-flight  instructors  than  inexperienced 
pilots.  The  same  conclusion  can  be  extrapolated  to  training  in  the  simu- 
lator. While  the  evidence  is  skimpy,  it  appears  that  even  personnel  with 
no  flight  experience  can  be  trained  to  be  effective  simulator  instructors. 

For  example,  in  a simulator  training  study  comparing  an  instructor  with 
several  thousand  hours  military  instructor-pilot  experience,  a recent 
flight  training  program  graduate,  and  a non-rated  individual  with  a few 

hours  dual  instruction  but  no  other  aeronautical  experience,  no  significant 

2 

differences  were  found  in  the  in-flight  performance  of  their  students. 


^Smode,  A.  F. , Hall,  E.  R.  and  Meyer,  D.  E.  An  Assessment  of  Research 
Relevant  to  Pilot  Training.  Technical  Report  AMRL-TR-66-196 , Aerospace 
Medical  Research  Laboratories,  Wright-Patterson  AFB,  OH,  1967. 

2 

Caro,  P.  W.,  Isley,  R.  N.  and  Jolley,  0.  B.  The  Captive  Helicopter 
as  a Training  Device:  Experimental Evaluation  of  a Concept.  Technical 

Report  68-9,  Human  Resources  Research  Organization,  Alexandria,  VA,  1968. 
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While  no  evidence  was  found  during  the  current  survey  to  refute 
those  findings,  they  clearly  were  inconsistent  with  the  views  of  many 
of  the  personnel  interviewed.  Both  proficiency  and  experience,  l.e., 
total  flight  time,  were  frequently  cited  as  having  significant  influence 
upon  simulator  training  effectiveness.  It  was  often  stated  that  trainees 
have  little  confidence  in  an  instructor  who  cannot  perform  well  himself. 

The  chief  value  of  instructor  experience  appears  to  be  a result  of 
the  fact  that  the  Air  Force  does  not  provide  comprehensive  simulator 
instructor  training,  so  the  experiences  of  the  older,  more  senior  instruc- 
tors enable  them  to  compensate  for  a lack  of  training.  Instructor  beha- 
viors that  were  cited  as  illustrative  of  effective  Instruction  could  be 
developed  in  all  instructors,  not  just  the  more  senior  ones,  through  an 
effective  program  designed  to  train  simulator  Instructors  and  to  standar- 
dize their  performance.  Examples  of  effective  Instructor  behaviors  cited 
during  the  study  included  (1)  the  best  instructor  does  not  try  to  teach 
all  he  knows,  but  teaches  what  he  has  needed  to  know  in  an  operational 
assignment;  (2)  the  good  instructor  uses  the  headset,  provides  ATC  simu- 
lation, and  does  other  things  to  simulate  situational  factors;  (3)  the 
good  instructor  lets  the  trainee  go  further  before  hitting  the  freeze 
button;  and  (A)  the  good  Instructor  concentrates  upon  the  needs  of  the 
trainee,  whereas  the  poor  instructor  concentrates  upon  getting  through 
the  syllabus. 

In  spite  of  the  fact  that  such  Instructor  behaviors  are  widely  viewed 
as  conducive  to  effective  simulator  training,  no  evidence  was  found  dur- 
ing the  survey  that  attempts  are  made  to  assure  that  those  behaviors  are 
standardized  among  simulator  Instructors.  The  training  provided  person- 
nel selected  to  become  simulator  Instructors  varies  from  base  to  base,  but 
in  no  instance  during  the  current  review  was  a simulator  instructor  train- 
ing program  noted  that  the  present  writer  would  judge  to  be  exemplary. 

The  training  provided  simulator  Instructors  varies  from  a "checkout"  on 
the  instructor’s  console  during  wliich  the  location  and  function  of 
switches  and  other  controls  are  illustrated  and  questions  are  answered  by 
an  already  qualified  Instructor,  to  a more  lengthy  "practice  teaching" 
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program  in  which  a newly  selected  instructor  may  spend  a week  or  more 
^ observing  simulator  training  and  actually  conducting  training  under  the 

supervision  and  with  the  assistance  of  a qualified  instructor.  In  addi- 
tion, since  the  personnel  involved  are  flight  qualified  and  often  train  in 
aircraft  as  well  as  in  the  simulator,  their  training  typically  includes 
an  extended  effort  to  raise  their  own  proficiency  and  to  standardize  their 
performance  in  the  aircraft  itself.  In  some  Instances,  such  training  in- 
cludes instruction  in  the  techniques  of  flight  training,  lesson  planning, 
and  preparation  of  instructional  material.  The  principal  deficiency  of 
these  courses,  however,  is  that  they  do  little  to  provide  the  Instructor 
specialized  knowledge  and  techniques  that  will  allow  him  to  employ  the 
simulator  effectively  and  efficiently  as  a training  vehicle  that  has 
unique  capabilities  different  from  those  of  the  aircraft. 

Existing  training  documents  related  to  the  use  of  simulation  as  in- 
structional tools  provide  very  little  guidance  to  the  instructor.  Typi- 
cally, they  provide  an  outline  of  what  tasks  will  be  performed  by  the 
pilot  trainee  during  a simulator  mission,  but  they  contain  little  or  no 
information  as  to  how  training  is  to  be  accomplished,  how  student  per- 
formance will  be  graded,  and  what  performance  criteria  apply  to  the  train- 
ees, given  their  particular  skill  and  experience  levels. 

Unfortunately,  not  all  simulator  instructors  are  well  prepared  for 
their  job,  however.  Hall,  Parker  and  Mever*  surveyed  a number  of  military 
training  programs  and  found  that  non-rated  enlisted  instructors  were  ill 
prepared  as  compared  with  pilots,  particularly  with  respect  to  relevant 
knowledge  of  the  aircraft.  They  also  noted  that  pilots  were  similarly 
ill  prepared  with  respect  to  knowledge  of  the  capabilities  and  limitations 
of  the  simulators.  Since  no  transfer  data  were  reported,  it  cannot  be 
determined  whether  this  factor  had  an  influence  upon  subsequent  in-flight 
performance  in  favor  of  either  instructor  group.  It  might  be  assumed, 

'Hall,  E.  R.  , Parker,  .1.  K.  , .Ir.  and  Mover,  I).  E.  A Study  ('f  Ai r 
Force  Flight  Simulator  Programs.  Technical  Report  AMRl,-TR-67-l  1 1, 
Aerospace  Ntdlcal  Research  Laboratories , Wr i ght-Pat terson  AFH,  OH,  1967. 
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however,  that  any  simulator  instructor  will  be  more  effective  if  he  is 
adequately  trained  in  the  use  of  the  simulator. 

Muckier,  .Nygaard,  O'Kelly,  and  Williams^  observed  that  in  some  cases 
a simulator  instructor  must  provide  supplementary  information  about  the  in 
flight  task  that  might  not  be  available  to  a non-rated  instructor,  thus 
presumably  tipping  the  scale  in  favor  of  pilots  as  simulator  instructors. 
Tliese  researchers  also  noted  that  instructor  ability  and  fidelity  of  simu- 
lation are  related  in  such  fashion  that  as  fidelity  increases,  the  neces- 
sary level  of  instructor  ability  may  decrease,  and  conversely,  as  fidelity 
decreases.  Instructor  ability  must  increase.  This  relationship  would  tend 
to  place  the  more  able  instructor  in  the  lower  fidelity  simulator  where  a 
greater  amount  of  supplementary  information  might  be  required.  it  has 
been  observed  that  just  the  opposite  situation  often  occurs.  The  more 
experienced  pilots  instruct  in  high  fidelity  simulators,  while  less  ex- 
perienced and  non-rated  personnel  instruct  in  older,  lower  fidelity 
devices . ^ 

Another  consideration  is  whether  there  should  be  one  instructor  or 
two  in  a simulator  training  program.  That  is,  is  simulator  training  effec 
tiveness  Influenced  by  whether  the  simulator  Instructor  is  also  the  air- 
craft flight  instructor?  While  this  variable  has  not  been  Isolated  for 
study,  there  appears  to  be  an  increase  in  effectiveness  when  a single 
instructor  is  responsible  for  both  simulator  and  aircraft  training.  Such 
an  arrangement  has  been  a feature  of  some  simulator  training  programs  for 
a number  of  years. ^ One  apparent  benefit  is  that  the  Instruction  given 

^Muckier,  F.  A.,  Nygaard,  J.  E. , O'Kelly,  1,.  I.  and  Williams,  A.  C., 

J r . Psychological  Varia b les  _i  nth  e Design  of  Flight  Simulator s_  for  Train- 
ing. Technical  Report  WADC  56-369,  Aerospace  Medical  Laboratory,  Wright- 
Patterson  AFR,  OH,  1959. 

2 

Caro,  P.  W.  Aircraft  Simulators  and  Pilot  Training,  Human  Factors, 
1973,  l_5,  502-509. 

3 

Williams,  A.  C.,Jr.  and  Flexman,  R.  E.  Evaluation  of  the  School 
Link  as  an  Aid  in  Primary  Flight  Instruction,  Aeronautical  bulletin  No.  5, 
university  of  Illinois  Institute  of  Aviation,  Savoy,  fL,  1949. 
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in  the  simulator  is  more  compatible  with  that  given  in  the  aircraft  when 
only  one  instructor  is  involved,  thus  reducing  any  potential  negative 
transfer  attributable  to  instructor-peculiar  performance  requirements. 

It  often  has  been  assumed  that  the  instructor  is  an  important  factor 
influencing  training  effectiveness,  and  such  may  well  be  the  case.  If  so, 
the  influence  must  be  attributable  to  the  manner  in  which  the  instructor 
functions,  i.e.,  to  the  degree  of  standardieatlon  in  his  administration 
of  the  training  program.  There  is  insufficient  evidence  available  at  this 
time  to  attribute  the  assumed  influence  to  instructor  experience  or  quali- 
fication per  se , assuming  he  has  undergone  an  instructor  training  program 
appropriate  to  the  Instructional  task  at  hand. 

Att itudes 

It  was  noted  during  the  current  survey  that  some  students  entering 
CCT,  particularly  those  coming  directly  from  Undergraduate  Navigator  Train- 
ing where  relatively  new  and  well  designed  simulators  are  used  extensively, 
have  quite  favorable  attitudes  toward  simulator  training.  If  they  are 
assigned  to  a Wing  that  employs  one  of  the  older,  "unrealistic"  simulators 
in  its  CCT,  their  favorable  attitudes  quickly  change,  even  to  the  point 
that  they  reportedly  dread  their  scheduled  simulator  training  missions. 
Instructors  in  those  CCT  squadrons  who  have  attempted  to  point  out  the 
relative  strength  of  the  simula'-or  training  programs  report  that  they 
feel  embarrassed  to  be  defending  equipment  that  the  students  know  to  be 
less  than  the  best,  and  these  same  instructors  believe  they  lose  credi- 
bility in  the  process.  This  situation  illustrates  an  apparent  relation- 
ship between  simulator  fidelity  and  trainee  attitudes  toward  simulator 
training. 

It  has  been  long  recognized  that  simulator  fidelity  has  an  impact 
upon  instructor  and  trainee  attitudes,  and  attitudes,  in  turn,  have  an 
Influence  upon  simulator  training  effectiveness.  Flexman^  described  this 

^Cited  in  Muckier,  F.  A.,  Nygaard,  J.  E. , O'Kelly,  L.  I.  and  Williams, 
A.  C.,  Jr.  Psychological  Variables  in  the  Design  of  Flight  Simulators  for 
Training.  Technical  Report  WAUC  56-369  , Aerospace  Medical  Laboratory, 
Wrlght-Patterson  AFB,  OH,  1959. 
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influence  as  follows  (p.  69): 


"Fidelity  of  simulation  can  operate  as  a 
motivational  variable.  If  the  simulator  looks,  acts,  feels  and  sounds  like 
the  airplane,  then  the  trainee  is  more  likely  to  be  convinced  that  prac- 
tice in  the  device  will  he  beneficial  to  him."  Spring^  reported  that  eli- 
minating platform  motion  during  training  in  an  air-to-air  combat  simulator 
was  viewed  by  the  pilots  as  causing  a significant  reduction  in  realism, 
and  "there  was  no  enthusiasm  for  the  simulation  with  this  limitation" 

(p.  4).  An  attempted  simulator  study  by  the  Coast  Guard  in  which  per- 
formance measures  were  being  compared  under  motion  and  no-motion  conditions 

had  to  be  terminated  because  the  participating  instructor  pilots  com- 

0 

plained  of  boredom  under  the  no-motion  condition.'’ 

In  circular  fashion,  attitudes  also  influence  simulator  design. 

3 

Williges,  Roscoe  and  Williges  noted  uhe  Importance  of  pilot  acceptance 
on  simulator  design  when  they  stated  that  decisions  to  include  complex 
and  expensive  motion  systems  in  simulators  are  invariably  determined  by 
pilots'  attitudes.  The  attitudes  of  pilots  toward  the  richness  of  visual 
displays  has  been  noted  earlier. 

The  most  direct  effect  of  trainee  a,  i instructor  attitudes  upon 
simulator  training  effectiveness  is  proba  i.y  upon  their  willingness  to 
engage  in  simulator  training  in  the  first  place.  Tnat  is,  devices  that 
are  viewed  favorably  seem  to  be  used  more  willingly  than  those  that  are 
viewed  less  favorably.  If  the  addition  of  a motion  system  or  visual 
display  to  a simulator  will  result  in  favorable  trainee  and  instructor 
attitudes  toward  simulator  training  and,  hence,  greater  utilization  of 
the  device,  it  is  possible  that  more  effective  simulator  training  will 

^Spring,  W.  G.  Advanced  Flight  Simulation  in  Air  Comhat  Training. 
Paper  presented  at  American  Institute  of  Aeronautics  and  Astronautics 
Visual  and  Motion  Simulation  Conference,  Davton,  OH,  April  26-28,  1976. 

2 

Personal  communication,  H.  K.  Povenmire,  U.  S.  Coast  Guard  Aviation 
Training  Center,  1976. 

3 

Williges.  B.  H.,  Roscoe,  S.  M.  and  Williges,  R.  C.  Synthetic  Flight 
Training  Revisited,  Human  Factors,  1973,  15,  541*560. 
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result  from  the  greater  utilization,  even  though  the  motion  and  visual 
features  per  se  may  contribute  nothing  directly  to  transfer. 

It  would  be  a mistake  to  attribute  all  favorable  attitudes  toward 
simulator  training  to  high  fidelity.  Although  informal  observation  sug- 
gests a high  correlation  exists  between  these  parameters,  there  are  rela- 
tively low  fidelity  devices  that  are  viewed  favorably  by  many  trainees 
and  instructors,  and  some  quite  sophisticated  devices  have  been  maligned 
unbearably  by  some  of  the  same  people.  One  relatively  low  fidelity  device 
used  extensively  bv  the  L’.  S.  .Armv  as  an  instrument  trainer  for  a number 
of  years  was  extolled  bv  the  device  instructors,  maligned  by  flight  in- 
structors, and  described  variously  as  a hindrance  or  an  aid  by  trainees. 

A study  of  the  effectiveness  of  training  conducted  in  the  device  was  less 
ambiguous:  the  training  was  useless.* 

A number  of  factors  other  than  simulator  fidelity  that  appear  to  be 
related  to  attitudes  toward  simulator  training  were  noted  during  the 
current  Air  Force  survey.  For  example,  an  apparent  relationship  was  noted 
between  expressed  attitudes  toward  simulator  training  and  the  proportion 
of  training  received  by  an  individual  in  a simulator.  While  no  attempt 
was  made  to  quantify  the  relationship,  it  was  the  writer's  observation  that 
attitudes  toward  the  worth  of  simulator  training  were  more  positive  in 
those  situations  in  which  the  limited  availability  of  aircraft  for  train- 
ing forced  greater  reliance  upon  simulation.  It  was  observed  that  pilots 
whose  access  to  aircraft  was  limited  to  a few  hours  per  month  spoke  more 
favorably  about  simulator  f lining  than  did  those  pilots  whose  duties 
Involved  more  frequent  flying.  Further,  WSOs,  whose  CCT  training  included 
less  flight  time  than  did  the  pilots  with  whom  they  were  paired,  also 
spoke  more  highly  of  the  simulator  than  did  the  pilots.  Finally,  it  was 
noted  that  aircrews  in  training  programs  that  involved  a relatively  high 
ratio  of  simulator-to-ai rcra f t training  time  appeared  more  favorably 


Isley,  R.  N.,  Caro,  P.  W.  and  Jolley.  0.  B.  Evaluation  of  Synthetic 
Instrument  Flight  Training  in  the  Officer-Warrant  Officer  Rotarv  Win 
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inclined  to  simulator  training  than  did  those  aircrews  undergoing  train- 
ing in  programs  involving  lower  simulator-to-aircraf t ratios. 

Another  factor  involved  the  general  intensity  of  simulator  training 
activities.  A tendency  was  noted  at  some  Bases  to  t rv  to  pack  as  much 
activity  as  possible  into  the  time  allotted  to  simulator  training.  At  one 
Base,  there  appeared  to  be  a negative  reaction  to  the  intensity  of  simu- 
lator training  that  was  reflected  in  the  attitudes  of  tlie  trainees.  In 
that  program,  the  trainees  expressed  dread  of  scheduled  training  in  the 
device,  explaining  that  training  events  (e.g.,  malfunctions)  occurred  so 
rapidly  that  unsatisfactory  performance  (as  judged  by  the  trainees  them- 
selves) was  almost  assured.  Simulator  training  for  them  was  a frustrating 
experience.  Although  they  acknowledged  probable  benefits  from  simulator 
training,  their  expressed  attitudes  toward  it  were  unfaborable.  In 
another  program  at  that  Base  using  the  same  device  but  in  a less  Intense 
training  program,  the  expressed  trainee  attitudes  were  much  more  positive. 

Another  factor  that  appears  to  influence  attitudes  toward  simulator 
training  is  the  reduced  "worth"  of  such  training,  when  compared  with 
similar  training  efforts  in  aircraft,  with  respect  to  certain  career 
goals.  For  example,  if  pilots  could  credit  simulator  training  time  toward 
the  hour  requirements  for  Flight  Leader  status,  they  would  tend  to  view 
simulator  training  more  favorably.  At  one  Base  included  in  the  current 
survey,  the  typical  squadron  pilot  reportedly  works  from  about  50  to  60 
hours  per  week.  Those  squadron  pilots  who  have  been  assigned  simulator 
instructing  duties  reportedly  work  an  additional  6 to  7 hours  per  week. 

The  attitude  expressed  by  some  of  these  simulator  instructor  pilots  was 
that  they  should  receive  some  credit  for  the  extra  time  spend  instructing 
in  the  simulator,  either  toward  career  goals  or  toward  meeting  their  own 
aircraft  and  simulator  minimum  training  requirements.  Without  such  credit, 
they  felt  at  a disadvantage,  and  they  expressed  somewhat  uncomplimentary 
views  about  simulator  training. 

The  attitude  of  simulator  instructors  probably  is  quite  important 
because  of  the  influence  most  instructors  have  over  their  students' 
attitudes.  It  was  found  during  the  current  survey  that  the  instructors' 


perceptions  of  their  assignment  to  instruct  in  simulators  varied  widely. 
Instructors  in  all  four  of  the  Cornmands  surveyed  reported  great  personal 
satisfaction  with  their  assignments  with  respect  to  the  opportunity  pro- 
vided for  professional  developmLnt , i.e.,  to  increase  their  job  knowledge 

with  respect  to  the  aircraft  for  whirh  they  were  training  students.  They 
invariably  felt  that  the  technical  knowledge  gained  while  serving  as  an 
instructor  better  qualified  them  for  their  next  operational  assignment. 

There  was  no  consensus,  however,  with  respect  to  the  likely  impact  of 
an  assignment  as  an  instructor  upon  their  career  as  an  Air  Force  officer 
and  aircrewman.  Within  at  least  one  Command,  such  an  assignment  was  viewed 
as  available  only  to  the  best  qualified  pilots,  and  the  assignment  was 
reported  to  be  actively  sought  for  its  favorable  Influence  upon  an  offi- 
cer's career  as  well  as  for  his  personal  satisfaction  and  professional 
development.  In  another  Command,  the  assignment  was  viewed  less  favorably 
career-wise,  and  it  was  reported  that  some  of  the  better  qualified  offi- 
cers seek  to  avoid  assignment  as  a simulator  and  flight  instructor  because 
of  its  low  status  and  because  it  is  not  perceived  as  particularly  bene- 
ficial to  one's  career.  Personnel  assigned  as  instructors  within  that 
Command  indicated  that  the  nature  of  the  assignment  made  all  the  instruc- 
tors appear  no  more  than  average  and  leads  to  lower  OERs  because  of  the 
poor  visibility  enjoyed  by  incumbents.  This  latter  comment  was  endorsed 
especially  by  non-pilot  instructors  whose  instructional  duties  are  con- 
fined almost  exclusively  to  the  simulator.  The  problem  of  visibility  was 
stated  thus  by  one  such  non-pilot  aircrewman  serving  as  a simulator  in- 
structor: "No  Colonel  is  going  to  observe  simulator  training." 

Within  all  Commands,  differences  were  found  in  the  ways  simulator 
instructors  were  treated  that  could  affect  their  attitudes  in  other  wavs. 
The  practice  at  Carswell  Air  Force  Base  of  carrying  all  new  simulator 
Instructors  to  the  American  Airlines  Flight  Academy  to  be  briefed  upon  the 
importance  of  simulator  training  at  American  appeare<i  to  boost  the  percep- 
tion of  these  instructors  with  respect  to  the  worth  of  their  jobs.  On 
the  other  hand,  the  practice  at  another  base  of  assigning  incoming  flight 
instructors  to  be  simulator  instructors  so  they  can  gain  more  knowledge  of 
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the  aircraft  while  waiting  for  a position  as  a squadron  flight  instructor 
to  become  available  probably  contributed  to  the  expressed  perception  that 
being  a simulator  instructor  is  something  to  do  until  an  opportunity 
arises  to  become  a "real"  instructor. 

Except  to  the  extent  that  favorable  attitudes  increase  device  use,  the 
effects  of  attitude  upon  simulator  training  appears  not  to  be  as  important 
as  one  might  think.  There  has  been  little  research  on  this  subject,  how- 
ever, probably  because  of  the  difficulty  in  controlling  attitudes  during 
experimentation.  In  a study  reported  by  Muckier,  Nygaard,  O'Kelly,  and 
Williams,^  negative  attitudes  toward  a trainer  were  induced  in  an  experi- 
mental group  by  stressing  the  device's  low  fidelity,  while  positive  atti- 
tudes were  induced  in  another  group  by  stressing  the  same  device's  train- 
ing effectiveness.  During  transfer  trials  in  the  aircraft,  both  groups 
were  found  to  have  benefited,  about  equally,  from  the  device  training, 
thus  indicating  that  the  induced  negative  attitude  did  not  affect  device 
training  effectiveness.  An  interesting  aspect  of  that  study  was  that 
the  negative  attitude  group  required  more  training  in  the  device  to  reach 
criterion,  thus  suggesting  an  effect  of  attitude  upon  training  efficiency 
rather  than  upon  training  effectiveness. 

No  transfer  study  was  found  during  the  current  literature  review  that 
indicated  that  attitude  per  se  was  a factor  influencing  simulator  training 
effectiveness.  On  the  basis  of  the  writer's  experiences,  it  appears  that 
just  the  reverse  may  be  the  case;  simulator  training  effectiveness  influ- 
ences attitudes  toward  simulator  training.  Abrupt  shifts  in  attitudes, 
particularly  among  instructors  and  training  program  managers  have  been 
observed  following  demonstrations  of  simulator  training  effectiveness. 

In  one  instance,  instructors'  very  negative  attitudes  toward  reduced  scale 
paper  mockups  of  a cockpit  became  favorable  when  they  discovered  that, 

*Mickler,  F.  A.,  Nygaard,  .1.  E.,  O'Kelly,  L.  I.  and  Williams,  A.  C.  , 
•Ir.  Psychological  Variables  in  the  Design  of  Flijdit  Simulators  for  Train- 
ing. Technical  Report  WADC  56-369  . Aerospace  Medical  Laboratory,  Wright- 
Patterson  AFB,  OH,  1959. 
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unknown  Co  Chem,  their  better  students  were  using  these  "devices"  on  their 
own.  In  a study  conducted  Jointly  by  the  Airline  Pilots  Association,  the 
Federal  Aviation  Agency,  United  Airlines,  and  the  U.  S.  Air  Force,  pilots' 
opinions  concerning  simulator  training  were  found  to  be  more  favorable 
following  their  participation  in  an  effective  simulator  training  program 
than  were  the  opinions  of  non-participating  pilots.^ 

In  spite  of  a lack  of  supporting  research  evidence,  there  is  a con- 
sensus among  trainees,  instructors  and  administrators  that  favorable  atti- 
tudes toward  simulator  training  increase  training  effectiveness.  This 
probably  is  correct,  at  least  in  the  sense  that  more  extensive  use  will  be 
made  of  simulators  if  they  arc  viewed  favorably.  It  may  be,  however,  that 
attitudes  are  Influenced  more  by  simulator  training  effectiveness,  than 
the  other  way  around.  A well  conducted  "test"  of  the  training  effective- 
ness of  a simulator  may  be  a very  influential  factor  in  assuring  that  its 
training  value  will  be  realized. 


Expectations 


Many  aviators  accept  the  proposition  that  training  in  a simulator 
might  be  helpful,  while  viewing  it  as  less  effective  than  training  in  an 
aircraft.  It  has  been  the  writer's  observation  that  simulator  training 
administered  under  the  control  of  such  individuals  never  exceeds  their 
expectations.  If  simulators  are  viewed  as  useful  only  as  procedures 
trainers  or  as  Instrument  trainers,  they  tend  to  be  used  only  as  procedures 
or  instrument  trainers,  even  though  the  same  devices  might  be  used  more 
effectively  by  others  who  view  them  as  offering  a greater  range  of  train- 
ing opportunities.  If  simvilators  are  viewed  as  useful  only  for  the 
initial  stages  of  the  development  of  a particular  skill,  to  be  followed 
bv  further  development  of  that  skill  in  the  aircraft,  simulator  training 
less  effective  tiian  if  simulators  are  viewed  as  substitutes  for  the  air- 
't  to  be  used  for  the  development  of  a particular  skill  to  criterion 
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before  transferring  to  the  aircraft.  '■Jliile  simulator  training  may  not 
always  prove  as  effective  as  some  might  expect,  expectations  appear  to 
place  a limit  upon  realized  effectiveness  by  limiting  the  manner  and  ex- 
tent of  simulator  training. 

Expectations  can  influence  simulator  training  effectiveness  in  more 
Subtle  ways  as  well.  The  expectation  that  a simulator  training  program 
will  prove  ineffective  can  influence  its  evaluation  in  the  expected 
direction.  Research  by  Rosenthal^  has  shown  that,  even  with  no  intention 
to  do  so,  an  experimenter  influences  the  outcome  of  his  research  in  the 
direction  of  his  expectations.  Since  many  "tests"  of  the  effectiveness 
of  simulator  training  are  conducted  by  pilots  who  hold  strong  views  con- 
cerning the  value  of  simulator  vs.  aircraft  training,  it  must  be  assumed 
that  their  expectations  can  and  sometimes  do  influence  the  test  data.  In 
those  Instances  in  which  there  is  real  or  perceived  pressure  from  a higher 
authority  to  reach  a particular  finding  concerning  the  utility  of  a par- 
ticular simulator,  the  effect  might  be  even  greater. 

There  is  an  almost  infinite  number  of  factors  that  might  shape  ex- 
pectations concerning  simulator  training  effectiveness.  One  factor  is 
prior  experience  with  simulator  training.  The  initially  favorable  atti- 
tude toward  simulator  training  of  recent  UNT  graduates  noted  above  was 
attributed  to  their  favorable  experiences  with  the  simulator  used  during 
UNT,  and  the  more  favorable  opinions  of  commercial  airline  pilots  toward 
simulators  was  attributed  to  their  participation  in  an  effective  simulator 
training  program.  Total  flight  time  is  probably  also  a factor,  since 
more  experienced  pilots  are  more  likely  to  have  had  unsatisfactory  experi- 
ences with  old  simulators  and  typically  put  greater  confidence  in  in- 
flight training. 

Miscel laneous  Factors 

There  are  a host  of  other  factors  that  can  be  presumed  to  influence 
simulator  training  effectiveness,  although  it  would  he  difficult  to 

^Rosenthal,  R.  Unintended  Communication  of  Interpersonal  Expectations, 
American  Behavioral  Sclent i^J^.,  19b7,  1_0,  24-26. 
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demonstrate  conclusively  that  any  of  them  is  significant  in  a particular 
instance.  For  example,  a factor  that  probably  has  had  a negative  influ- 
ence upon  the  effectiveness  of  one  or  more  of  the  Air  Force  simulator 
training  activities  included  in  the  present  survey  is  the  resistance  put 
forth  by  some  of  the  personnel  involved  to  the  idea  of  using  simulators 
for  training  rather  than  aircraft.  Training  in  simulators,  particularly 
for  many  of  the  more  senior  aircrews,  represents  a significant  change 
from  well  established  and  well  accepted  training  practices,  and  changing 
from  one  practice  to  another  has  often  been  resisted  by  the  personnel 
involved  for  no  reason  other  than  that  it  is  different. 

A review  of  the  research  literature  on  the  process  of  effecting 
change,  such  as  substituting  simulators  for  aircraft  in  aircrew  training 
as  has  occurred  in  the  Air  Force  in  recent  years,  has  been  prepared  by 
McClelland.'  In  that  review,  McClelland  states  that  the  Introduction  of 
a new  training  device  is  not  in  Itself  sufficient  to  assure  its  acceptance 
by  the  personnel  expected  to  use  it.  Rather,  time,  money  and  continuous 
effort  are  required  to  assure  its  effective  use. 

A number  of  the  Air  Force  programs  surveyed  reflected  efforts  presumably 
intended  to  overcome  some  of  the  resistance • suspected  to  have  resulted 
from  changing  from  aircraft  to  simulators.  For  example,  as  a means  of 
emphasizing  the  value  of  simulator  training,  several  of  the  Commands  are 
requiring  that  the  simulators  in  subordinate  units  be  used  for  training  at 
relatively  high  rates,  such  as  80  hours  per  week.  Such  a requirement  may 
well  have  an  eventual  favorable  influence  upon  attitudes  toward  changing 
to  simulators.  In  the  meantime,  however,  the  requirement  that  a simulator 
be  used  for  a specified  number  of  hours  per  week  increases  the  likelihood 
that  at  least  some  simulator  training  will  be  ineffective.  Since  such  a 
requirement  is  essentially  unrelated  to  training  needs  or  to  numbers  of 
personnel  to  be  trained,  keeping  the  simulator  busy  sometimes  involves 

'McClelland,  W.  A.  The  Process  of  F.ffecting  Change.  Professional 
Paper  32-68,  Human  Resources  Research  Office,  Alexandria,  VA,  October 
1968. 
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repeating  training  missions  or  exercises  that  already  have  been  mastered. 
Further,  at  one  Base  where  such  a requirement  existed  at  the  time  of  the 
survey,  there  were  reports  of  Instances  in  which  the  ratio  of  time  spent 
in  the  simulator  to  time  spent  in  simulator  briefing  and  debriefing  ac- 
tivities decreased  although  all  such  time  was  recorded  as  simulator  train- 
ing time.  It  was  also  reported  that  pilots  (often  of  junior  ranks)  who 
were  being  required  to  use  up  the  scheduled  simulator  time  were  sometimes 
already  proficient  at  tasks  which  could  be  practiced  in  the  devices. 

A problem  commonly  believed  by  the  Air  Force  personnel  Interviewed 
to  affect  simulator  training  effectiveness  was  inadequate  maintenance. 

This  problem  was  particularly  apparent  among  some  of  the  older  simulators 
for  which  spare  parts  were  difficult  to  obtain.  Pilots  using  those  older 
simulators  described  instances  in  which  planned  syllabus  training  activi- 
ties could  not  be  completed  or  were  performed  in  degraded  fashion  because 
a portion  of  the  simulator  (e.g.,  the  motion  platform  or  an  avionics 
component)  was  out  of  service.  l-Ihile  it  is  sometimes  necessary  to  cancel 
a simulator  training  period  because  of  simulator  failures,  the  more  common 
practice  is  to  complete  the  scheduled  training  period  using  a generally 
unsatisfactory  simulator  (and  presumably  deriving  fewer  training  benefits) 
because  of  the  punishing  consequences  of  having  to  reschedule  a lost 
training  period  at  undesirable  times,  such  as  on  weekends.  The  schedule 
of  training  use  of  most  of  the  simulators  included  in  the  survey  did  not 
Include  spare  time  during  duty  hours  for  rescheduling  lost  training  periods. 

In  situations  in  which  an  appropriate  simulator  is  not  available,  it 
is  sometimes  necessary  to  make  ineffective — or  at  least  inefficient — use 
of  the  simulators  which  are  available.  For  example,  two  C-5  simulators 
are  available  for  Continuation  Training  at  Travis  Air  Force  Base,  but 
part-task  devices  are  not  available  for  the  separate  training  of  the  vari- 
ous crew  members,  e.g.,  navigator  or  flight  engineer.  One  of  the  C-5 
simulators  is  used  for  pilot  and  co-pilot  training  during  the  first  four 
days  of  the  five-day  Continuation  Training  program,  while  the  other  simu- 
lator is  used  for  navigator  and  engineer  training.  The  full  crew  is 
trained  together  in  a single  simulator  on  the  fifth  day.  While  use  of  the 
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full  simulator  for  training  only  a portion  of  the  flight  crew  cannot  be 
viewed  as  optimum  from  the  efficiency  standpoint,  the  factor  influencing 
this  approach  is  the  availability  of  the  simulator  and  the  non-availability 
of  part-task  devices.  Alternative  and  possibly  more  effective  solutions 
would  be  to  have  simulators  more  appropriately  designed  to  the  training 
requirement  or  to  develop  training  programs  which  would  permit  training  of 
entire  crews  in  a single  simulator. 

Finallv,  the  presence  or  absence  of  a simulator  instructor  was  sug- 
gested at  one  Base  as  a likely  influence  upon  simulator  effectiveness.  At 
that  Base,  some  simulator  missions  are  graded  while  others  are  not,  and  a 
qualified  Instructor  is  required  to  be  present  only  during  graded  flights 
(this  particular  device  is  normally  manned  by  both  a simulator  Instructor 
and  a non-rated  simulator  operator  who  has  no  instructional  responsibili- 
ties). Several  pilots  undergoing  Continuation  Training  there  expressed  the 
opinion  that  the  absence  of  an  Instructor  during  certain  training  flights 
deprived  them  of  critiques  that  might  increase  the  effectiveness  of  the 
training  they  were  receiving. 

While  the  absence  of  an  instructor  in  that  particular  training 
program  may  have  reduced  the  effectiveness  of  the  training,  it  is  not 
necessarily  true  that  an  instructor  must  always  he  present  for  simulator 
training  to  be  effective.  A recent  study  by  Tyler,  McFadden,  Eddowes, 
and  Fuller^  compared  Che  effectiveness  of  simulator  training  in  an  instru- 
ment training  program  in  HPT  under  experimental  conditions  which  included 
tutoring  by  a qualified  flight  instructor  and  selt-lnsCructlon  in  which 
case  no  instructor  was  present.  They  found  that,  for  the  training  ob- 
jectives under  consideration,  the  participation  of  the  instructor  added 
nothing  to  the  effectiveness  of  the  simulator  training.  It  Is  likely  that 
there  may  even  be  positive  benefits  associated  with  removing  the  instructor 
during  selected  training  activities  in  some  situations,  since  it  has  been 

Tyler,  D.  M. , McFadden,  R.  W. , Eddowes,  E.  E. , and  Fuller,  R.  R. 
Investigation  of  Diagnostic,  Error  Detector,  and  .Self-Taught  Instructional 
Strategies  for  Flight  Simulator  Programs.  Technical  Report  AFHRI.-TR-76-65 , 
USAF  Human  Resources  Laboratory,  Williams  AFB , AZ,  October  1976. 
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observed  to  lead  to  problem-solving  behavior  that  may  not  be  easily  elicited 

1^  when  an  instructor  is  present.^ 

COMMENT 

I The  foregoing  discussion  of  factors  noted  during  the  current  survey  of 

I Air  Force  simulator  training  programs  is,  of  course,  largely  speculative. 

1 Since  there  was  no  objective  evidence  concerning  the  effectiveness  of  any 

I of  the  simulator  training  programs  surveyed,  there  could  be  no  direct  evi- 

dence that  their  effectiveness  was  attributable  to  any  one  or  more  of  the 
factors  discussed  herein.  The  effectiveness  of  Air  Force  simulator  traln- 
I ing  undoubtedly  is  subject  to  many  obvious,  as  well  as  subtle,  influences, 

and  it  is  quite  likely  that  many  of  the  factors  involved  Interact  in  unex- 
pected and  possibly  unpredictable  ways. 

It  should  be  noted  that  the  Air  Force  currently  is  engaged  in  a con- 
tinuous program  of  revising  its  simulator  training  activities.  Awareness 
of  the  probable  influence  upon  simulator  training  effectiveness  of  factors 
such  as  those  identified  above  was  apparent  on  the  part  of  many  of  the 
personnel  Involved  in  those  programs.  Deliberate  efforts  were  noted  at 
I many  Bases  to  improve  simulator  training  by  improving  device  fidelity, 

t 

I selecting  qualified  instructors.  Influencing  attitudes  and  expectations, 

I 

and  manipulating  many  of  the  other  factors  that,  either  through  consensus 
1 or  evidence  from  the  research  literature,  are  believed  to  influence  simu- 

lator training  effectiveness.  It  is  unfortunate  that  firm  evidence  does 
1 not  exist  as  to  how  to  manipulate  the  various  Influences  in  combination 

I to  produce  the  optimum  simulator  training  program. 


Caro,  P.  W.  , Isley,  R.  N.  and  Tolley,  0.  B.  Mission  .Suitability 
Testing  of  an  Aircraft  Simulator.  Technical  Report  75-12,  Human  Resources 
j Research  Organization,  Alexandria,  VA,  Tune  1975. 
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IV.  RECOMMENDATIONS 


INTRODUCTION 

The  project  reported  here  was  undertaken  in  an  attempt  to  gain  a better 
understanding  of  how  simulator  training  might  be  made  more  effective  in  the 
Air  Force.  To  that  end,  a number  of  factors  that  influence  the  effectiveness 
of  such  training  have  been  identified  through  surveys  of  representative  Air 
Force  simulator  training  activities  and  through  reviews  of  relevant  research 
and  technical  literature.  In  some  instances  it  is  likely  that  merely  calling 
attention  to  these  factors  will  result  in  their  being  manipulated  by  personnel 
responsible  for  simulator  training  programs,  and  the  effectiveness  of  those 
programs  may  be  enhanced  as  a result.  To  the  extent  that  this  does  occur,  the 
Air  Force  will  benefit  directly  from  the  Information  developed  during  this 
study. 

In  other  instances,  however,  the  information  about  simulator  training  may 
not  be  such  that  it  can  be  used  directly  by  personnel  involved  in  that  train- 
ing. For  example,  use  of  some  of  the  information  may  be  precluded  by  admini- 
strative considerations  beyond  the  control  of  those  who  design  simulators  or 
those  who  conduct  simulator  training.  Further  research  may  be  required  to  deter- 
mine the  significance  or  even  the  accuracy  of  some  of  the  information  involved, 
or  to  indicate  how  the  factors  identified  during  the  project  might  be  manipu- 
lated to  best  advantage.  The  need  for  administrative  action  and  further  research 
on  simulator  training  effectiveness  is  the  subject  of  Section  IV  of  this  report. 

RECOMMENDATIONS 

The  recommendations  which  follow  are  derived  from  the  information  developed 
during  this  project  and  presented  in  Sections  II  and  III  of  this  report.  These 
recommendations  are  for  both  administrative  action  by  the  Air  Force  and  for 
further  research  and  development  activities.  The  Intent  of  each  recommendation 
is  to  increase  the  effectiveness  of  present  and  future  Air  Force  simulator 
training  activities  or  to  develop  Information  that  will  lead  to  increased  simu- 
lator training  and/or  cost  effectiveness  in  the  future. 

The  recommendations  are  presented  below  in  an  order  that  parallels  the 
presentation  of  related  information  in  Sections  II  and  III  of  this  report. 
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Thus,  recommendations  related  to  determining  simulator  training  effectiveness 
are  presented  first,  followed  by  those  related  to  the  various  factors  influ- 
encing simulator  training  effectiveness  which  are  discussed  in  the  report. 

This  ordering  was  selected  for  the  convenience  of  the  reader  who  may  wish  to 
review  earlier  discussions  related  to  specific  recommendations. 

Determining  Simulator  Training  Effectiveness 

In  general,  too  little  emphasis  has  been  placed  upon  validation  of  simu- 
lator training  programs  in  the  Air  Force.  Validation  efforts  that  have  taken 
place  have  tended  to  be  unsystematic  in  nature,  to  be  made  by  personnel  un- 
trained in  evaluation  methodologies,  and  to  rely  heavily  upon  the  subjective 
judgment  of  personnel  who  are  untrained  in  the  making  of  such  judgments  or 
who  may  have  vested  interests  in  the  outcome  of  the  validation  studies.  It 
is  recommended  that  emphasis  be  placed  at  all  command  levels  upon  validating 
each  simulator  training  activity,  employing  validation  study  design  models  that 
emphasize  objective  measurement  of  trainee  performance  against  predetermined 
performance  standards  in  the  operational  aircraft.  Technical  reports  docu- 
menting each  such  study's  findings  and  the  methods  employed  in  conducting  the 
study  should  be  a formal  requirement  of  each  validation  effort. 

It  is  recognized  that  the  Air  Force  may  not  always  have  adequately  trained 
personnel  available  to  design,  conduct  and  report  simulator  training  effective- 
ness validation  studies.  Therefore,  it  is  recommended  that  a program  be  ini- 
tiated to  identify  or  develop  the  resources  necessary  to  accomplish  these 
studies.  Such  a program  should  Include  special  training  for  personnel  respon- 
sible for  simulator  training  program  validation,  the  provision  of  consultants 
(either  Air  Fotce  or  contract  personnel)  to  assist  in  the  design  and  conduct 
of  the  studies,  and  the  development  of  detailed  procedural  manuals  to  be 
followed  in  the  studies'  conduct  and  reporting. 

Simul ator  Design — Design  for  Tr aj. ning 

Air  Force  simulators  are  based  upon  various  design  concepts,  most  of 
which  appear  to  be  more  closely  related  to  reproduction  of  features  of  the  air- 
craft than  to  provision  of  an  environment  optimized  for  training.  This  is  not 
surprising,  since  design  guidelines  tliat  relate  specific  simulator  features  to 
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training  objectives  and  processes  often  do  not  exist.  Fundamental  design 
problems,  such  as  the  location  and  functions  of  instructing  personnel,  appear 
to  have  been  resolved  in  the  design  of  many  Air  Force  simulators  in  a manner 
which  emphasized  fidelity  of  simulation  of  the  aircraft.  The  emphasis  in  the 
design  of  some  Air  Force  simulators  appears  to  have  been  only  secondarily  upon 
their  training  purposes. 

It  is  recommended  that  a systematic  program  of  research  be  initiated  by 
the  Air  Force  that  will  examine  simulator  design  considerations  as  a function 
of  specified  training  objectives  and  of  the  manner  in  which  tlie  devices  are  to 
be  used  to  achieve  those  objectives.  Emphasis  in  such  research  should  be  upon 
defining  the  roles  of  the  instructor  and  other  personnel  Involved  in  the  in- 
structional process  (e.g.,  other  aircrew  members)  and  upon  identifying  training 
processes  that  could  be  employed  to  enhance  simulator  training  effectiveness 
and  efficiency.  Design  concepts  should  be  sought  that  will  emphasize  the 
simulators'  training  purposes  rather  than  the  characteristics  of  the  aircraft 
being  simulated.  Such  research  should  incorporate  efforts  in  which  existing 
Air  Force  simulators  are  examined  to  determl'ne  the  limitations  their  various 
designs  place  upon  the  manner  of  their  use  as  well  as  efforts  to  develop  and 
test  alternative  simulator  design  concepts  for  various  training  requirements. 
This  research  should  attend  to  all  simulator  design  features  which  can  Impact 
training  effectiveness.  Examples  of  such  features  include  overall  simulator 
configuration  and  manning  requirements  as  well  as  the  provision  of  instructional 
aids  such  as  record-playback  and  automatic  adaptation  of  program  difficulty  to 
trainee  error  rates. 

Simulator  Design — Visual  Fidelity 

Simulator  design  is  undergoing  a major  change  with  respect  to  visual  simu- 
lation. Whereas  only  one  of  the  simulators  examined  during  the  present  survey 
Incorporated  an  extra-cockpit  visual  display,  many  future  Air  Force  aircraft 
simulators  will  have  such  displays  or  will  be  designed  so  that  such  displays 
can  be  added.  Presumably,  the  costs  to  the  Air  Force  of  these  additions  to 
future  simulators  are  justified  because  of  the  increased  training  value  they 
will  provide.  It  should  be  noted,  however,  that  this  presumption  is  neither 
supported  nor  refuted  by  presently  available  research  findings.  Instead,  the 
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presumption  is  supported  by  a desire  to  increase  the  physical  correspondence 
between  the  simulator  and  the  aircraft  training  environment  in  the  area  of 
visual  cues,  and  thus  to  increase  the  apparent  scope  of  training  for  which  the 
device  is  suitable. 

Present  day  visual  display  technologies  are  limited,  and  display  costs  are 
high.  Therefore,  the  need  for  visual  displays  should  be  examined,  and  the 
training  effectiveness  of  existing  simulators  without  visual  displays  for  skills 
involving  extra-cockpit  visual  cues  should  be  investigated.  The  extent  to  which 
presently  existing  Air  Force  simulators  without  visual  displays  can  be  used 
to  meet  visual  training  objectives — or  at  least  to  reduce  the  time  required 
in  the  aircraft  to  meet  those  objectives — has  not  been  established.  There  are 
probably  a numbef'>!<  visual  training  objectives  which  are  dependent  upon  extra- 
cockpit visual  simulation,  but  it  is  also  possible  that  many  that  presently 
are  presumed  to  be  so  dependent,  in  fact,  are  not. 

Research  to  find  alternatives  to  expensive  extra-cockpit  displays,  such 
as  displays  inside  the  simulator  cockpit  and  cognitive  pre-training  conducted 
in  the  classroom,  also  would  appear  to  offer  possible  visual  training  benefits 
at  relatively  modest  costs.  For  many  visual  training  needs,  an  extra-cockpit 
display  simply  may  not  be  necessary.  For  others,  of  course,  it  may  be  essen- 
tial. 

It  is  recommended  that  a program  of  visual  simulation  research  be  under- 
taken in  which  attention  is  directed  to  the  cognitive  and  visual  cues  essential 
to  the  attainment  of  the  visual  training  objectives  associated  with  various 
aircraft  systems  and  missions.  Emphasis  in  such  research  should  be  upon  analy- 
tic efforts  directed  to  the  tasks  to  be  trained  and  upon  the  development  of 
simple  models  and/or  analogs  of  visual  scenes  consisting  of  essential  cues 
which  can  be  displayed  inexpensively  to  the  simulator  pilot.  The  goal  should 
not  be  simply  to  reproduce  a particular  visual  scene,  or  even  to  produce  a 
"low  fidelity"  representation  of  tliat  scene,  but  to  determine  the  cognitive 
and  visual  cues  essential  to  the  elicitation  of  training  objectives. 

Present  visual  display  research,  directed  primarily  to  the  development  of 
techniques  of  image  generation,  storage,  and  display,  should  not  be  affected 
by  the  conduct  of  the  research  recommended  above. 
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simulator  Design — Motion  Fidelity 

The  recent  reports  that  performance  In  the  aircraft  may  not  benefit  from 
the  presence  of  cockpit  motion  were  welcomed  by  many  who  have  been  concerned 
about  the  costs  of  procuring,  maintaining,  and  housing  modern  platform  motion 
systems.  In  some  instances,  the  limited  scope  of  these  recent  motion  studies 
may  have  been  overlooked  because  of  the  potential  savings  their  results  would 
permit.  Since  these  studies'  results  are  not  completely  consistent  with  the 
results  of  other  simulator  motion  research,  however,  the  inconsistencies  should 
be  examined  carefully. 

The  distinction  between  maneuver  and  disturbance  motion  provides  a con- 
venient conceptual  framework  within  which  to  examine  the  apparent  diffe-ence 
between  findings  of  the  earlier  research  on  motion  and  the  Jacobs  and  Roscoe 
and  the  ASPT  studies'  findings  that  motion  may  not  be  important  in  training. 

It  is  clear  that  in  the  studies  in  which  motion  did  not  appear  to  influence 
pilot  performance,  the  motion  involved  was  predominantly,  if  not  exclusively, 
of  the  maneuver  variety.  On  the  other  hand,  disturbance  motion  was  the  pre- 
dominant type  of  motion  in  studies  in  which  a contrary  finding  was  suggested. 
Thus,  it  appears  that  the  results  of  both  sets  of  studies  can  be  accepte 
and  attributed  to  the  limited  nature  of  the  motion  simulation  involved  in  each. 
Disturbance  motion  is  important,  at  least  in  training  situations  where  dis- 
turbance cues  are  important  and  when  the  aircraft  simulated  is  unstable  or 
particularly  responsive  to  control  input.  Maneuver  motion  may  be  important 
also  under  some  of  these  circumstances,  but  the  evidence  available  at  this 
time  has  not  shown  that  it  contributes  to  transfer  of  training. 

Because  the  distinction  between  maneuver  and  disturbance  motion  has  only 
recently  been  articulated,  there  has  been  little  opportunity  to  examine  sys- 
tematically the  influence  of  each  upon  simulator  training  effectiveness.  Most 
prior  training  research  on  motion  appears  to  have  dealt  primarily  with  maneuver 
motion,  and  maneuver  motion  appears  reasonably  well  represented  in  the  newer 
simulators,  although  time  lags  between  pilot  manipulation  of  aircraft  controls 
and  motion  system  responses  have  been  a major  problem  in  some  of  them.  Dis- 
turbance motion  has  been  less  thoroughly  investigated,  and  is  poorly  repre- 
sented in  some  newer  simulators. 

Additional  research  upon  the  role  of  disturbance  motion  in  training  is 
clearly  needed,  and  a program  of  such  research  is  recommended.  Emphasis  in 
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the  research  should  be  three-iold:  (1)  analysis  of  disturbance  motion  cues 

associated  with  specific  simulator  training  objectives;  (2)  development  of 
models  for  the  representation  of  critical  components  of  such  motion  In  simula- 
tors (e.g.,  rates  of  onset,  direction  of  movement,  and  duration);  and  (3)  de- 
termination of  the  effects  upon  transfer  of  training  of  the  presence  and  absence 
of  such  motion.  Because  of  the  continuing  concern  over  the  costs  afsoclated 
with  motion  simulation,  future  research  on  motion  simulation  should  also  examine 
the  use  of  motion  systems  with  limited  axes  of  motion,  g-suits  and  seats,  and 
"seat  shakers"  to  determine  whether  the  disturbance  cues  found  to  be  important 
In  training  can  be  represented  adequately  in  such  relatively  low  cost  motion  | 

devices.  In  any  event,  future  motion  system  designs  should  be  responsive  to  j 

requirements  to  provide  specific  movements  which  cue  specific  pilot  responses  j 

rather  than  to  provide  motions  which  simply  correspond  to  motions  of  the  simu-  j 

lated  aircraft. 

Simulator  Design — Handling  Characteristics 

Aircraft  response  and  handling  characteristics  received  significant  re- 
search attention  several  decades  ago  when  then-available  analog  simulation 
techniques  made  faithful  simulation  of  an  aircraft  both  difficult  and  expensive. 

Present  day  technologies  are  such  that  close  approximations  to  almost  any  air- 
craft flight  characteristic  can  be  achieved  in  a simulator  at  relatively  modest 
costs.  Therefore  emphasis  has  shifted  from  research  to  establish  the  necessary 
minimum  levels  of  fidelity  of  handling  characteristics  required  to  assure  simu- 
lator training  effectiveness,  to  the  development  of  digital  computer  modeling 
and  programming  techniques  which  Improve  such  fidelity.  Major  Investment  of 
training  research  resources  in  simulator  handling  characteristics  research  does 
not  appear  warranted  at  this  time. 

Training  Programs 

While  it  is  generally  acknowledged  that  the  manner  in  which  a simulator 
is  used  limits  its  training  effectiveness,  relatively  little  attention  has  been 
devoted  to  the  development  of  optimum  ways  of  using  these  devices.  Except  to 
the  extent  that  general  learning  concepts  may  be  applied  to  the  simulator 
training  situation,  few  research-based  guidelines  exist  for  the  simulator 
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training  program  developer  to  follow  in  establishing  his  training  program. 

An  obvious  recommendation  from  the  present  study  is  that  more  effort  be  de- 
voted to  research  on  how  to  manipulate  variables  believed  to  affect  learning 
in  a simulator  training  situation. 

It  did  not  appear  to  the  writer  that  the  effectiveness  of  a majority  of 
the  simulator  training  programs  included  in  the  present  survey  was  constrained 
by  a lack  of  research,  however.  Instead,  instances  were  noted  in  which  prac- 
tices did  not  make  full  use  of  available  information  about  human  learning  and 
performance.  Training  programs  in  which  knowledge  of  results  and  reinforce- 
ment were  withheld  until  training  was  completed,  for  example,  would  not  appear 
to  be  designed  around  current  understandings  of  learning  processes.  It  is  be- 
lieved that  a thorough  review  of  each  Air  Force  simulator  training  program  by 
personnel  whose  expertise  lies  in  the  area  of  human  learning  and  performance 
would  provide  numerous  suggestions  for  program  development  and  modification  to 
be  considered  by  those  whose  areas  of  expertise  lie  in  operational  flying  and 
flight  training.  Such  reviews  are  recommended. 

The  Air  Force  currently  is  redefining  its  simulator  training  activities 
through  the  application  of  the  Instructional  System  Development  (ISO)  process. 
It  is  expected  that  these  ISD  efforts  will  have  a significant  impact  upon  simu- 
lator training,  particularly  with  respect  to  the  content  of  simulator  training. 
It  is  recommended  that  these  efforts  be  encouraged  and  expedited  where  possible 
in  both  combat  crew  training  and  continuation  training  programs. 

Personnel  - Trainees 

The  Air  Force  and  other  training  agencies  have  invested  heavily  in  re- 
search to  develop  an  effective  means  of  selecting  aircrew  trainees.  Numerous 
variables  which  predict  the  effectiveness  of  flight  training  have  been  identi- 
fied, and  many  of  these  same  variables  probably  predict  simulator  training 
effectiveness  equally  well.  Since  personnel  selected  for  aircrew  training 
have  all  been  screened  to  eliminate  those  for  whom  initial  flight  and  simu- 
lator training  programs  could  be  predicted  to  be  ineffective,  it  should  not 
be  expected  that  factors  would  be  found  that  would  affect  undergraduate  level 
simulator  training  effectiveness  differentially  for  different  groups  of 
selectees.  The  selection  process  has  assured  that  aircrew  trainees  are  homo- 
geneous with  respect  to  those  factors  that  might  be  influential  in  that  regard. 
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On  the  other  hand,  it  might  be  expected  that  simulator  training  effective- 
ness— and  flight  training  effectiveness — might  well  be  related  to  flight  ex- 
periences gained  by  aircrews  subsequent  to  completion  of  the  selection  process, 
since  these  experiences  are  known  to  vary  widely.  The  findings  of  the  present 
survey  and  literature  search  do  not  support  that  expectation.  Instead,  it  was 
found  that  simulator  training  can  be  effective  for  the  neophyte  as  well  as  for 
the  highly  experienced  pilot.  However,  these  findings  do  suggest  that  simu- 
lators should  be  used  differently  for  aircrews  with  different  levels  of  ex- 
perience if  simulator  training  is  to  be  equally  effective  for  all.  Therefore, 
it  is  recommended  that  more  emphasis  be  placed  upon  identifying  the  needs  of 
the  individual  trainee  in  Air  Force  simulator  training  programs  than  was  typi- 
cally noted  during  the  present  survey.  Rather  than  having  standard  simulator 
training  syllabi  for  each  aircrew  position  which  all  assigned  personnel  must 
complete,  simulator  training  would  be  more  effective  if  individual  skill  defi- 
ciencies were  diagnosed,  and  simulator  training  content  and  duration  were 
tailored  to  those  individual  needs.  A training  system  that  does  not  recognize 
differences  among  trainees  is  an  inefficient  system  at  best.  Its  chief  virtue 
lies  in  the  relative  ease  with  which  it  may  be  administered. 

Personnel  - Instruct qr^ 

The  role  of  the  instructor  in  Air  Force  simulator  training  programs  has 
not  been  well  defined.  Most  often  in  the  programs  surveyed,  the  instructor  was 
viewed  as  an  expert  aircrewman  who,  because  of  his  competence  in  the  cockpit  of 
an  aircraft,  knows  what  to  do  and  how  to  do  it  in  order  to  train  students  in 
a simulator.  Just  what  it  is  that  he  does  that  makes  him  a good  simulator  in- 
structor was  not  clear,  however.  Wliile  there  was  unanimous  agreement  that  some 
simulator  instructors  are  better  than  others,  there  were  differences  in  the  way 
good  instructors  were  characterized. 

It  is  recommended  that  attention  be  devoted  to  the  specification  of  cri- 
teria for  the  simulator  instructor,  i.e.,  what  does  a good  instructor  do  that 
distinguishes  him  and  his  students  from  other  instructors  and  their  students? 

It  is  believed  that  the  inattention  to  simulator  instructor  training  noted 
during  the  current  survey  results  from  the  fact  that  most  of  the  personnel 
interviewed  were  not  sure  what  the  content  of  such  training  should  be.  If 
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training  effectiveness  and  efficiency  are  related  to  the  skills  and  techniques 
of  the  simulator  instructor,  then  it  is  Important  that  personnel  selected  for 
such  assignments  be  provided  the  training  that  will  permit  them  to  develop  such 
skills  and  techniques.  Before  such  training  can  be  provided,  those  skills  and 
techniques  must  be  identified.  It  is  also  recommended  that  simulator  instruc- 
tor training  programs  be  developed  to  provide  the  needed  training  as  soon  as 
the  requirements  of  that  training  can  be  identified  and  validated. 

Attitudes,  Expectations  and  Miscellaneous  Factors 

It  has  been  the  writer's  experience  that  few  aircrewmen  can  be  persuaded 
that  simulator  training  is  effective.  Instead,  they  must  be  shown.  A well 
designed,  conducted  and  documented  test  or  study  that  demonstrates  that  a par- 
ticular simulator  training  program  is  effective  can  have  favorable  influences 
upon  trainee  attitudes  and  expectations  toward  such  training.  The  recommendation 
made  earlier  in  this  report  that  validation  studies  be  conducted  of  all  Air 
Force  simulator  training  activities  is  underscored  here,  because  such  studies 
can  provide  the  effectiveness  demonstrations  needed  to  increase  willing  par- 
ticipation in  simulator  training.  These  demonstration  studies  are  of  particular 
importance  in  Instances  in  which  a elmulator  may  lack  a dimension  of  fidelity, 
e.g.,  platform  motion,  that  is  viewed  as  important  by  the  trainees  who  must 
use  the  device. 

It  is  also  recommended  that  personnel  responsible  for  the  oversight  of 
simulator  training  in  the  Air  Force  examine  some  of  the  administrative  practices 
associated  with  that  training.  Questions  should  be  raised  on  both  a command  and 
a local  level  as  to  whether  any  such  practices  are  likely  to  affect  simulator 
training  attitudes  and  expectations  in  negative  ways.  A number  of  suspect 
practices  were  noted  during  the  present  survey:  training  schedules  that  are 

unduly  lengthy  or  interfere  unnecessarily  with  off-duty  activities;  failure  to 
give  an  aircrewman  credit  toward  career  goals  for  simulator  training  accomplish- 
ments when  such  credit  is  readily  available  for  comparable  performance  in  air- 
craft; and  awarding  a lower  status  to  simulator  instructor  assignment  than  to 
"real"  or  in-flight  Instructor  assignment.  Systematic  examination  of  such 
administrative  practices  and  restrictions  could  lead  to  more  favorable  attitudes 
toward  simulator  training. 
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An  examination  of  administrative  practices  may  also  identify  factors  that 
might  influence  simulator  training  effectiveness  more  directly.  Practices  that 
emphasize  training  processes  such  as  syllabus  completion  and  high  levels  of  per- 
sonnel activity  instead  of  training  obiectives  do  not  necessarily  lead  to  highly 
effective  training.  Objective  examination  of  the  needs  for  such  practices  might 
lead  to  the  Identification  of  alternatives  which  would  be  more  beneficial. 

COMMENT 

Our  understanding  of  factors  influencing  the  effectiveness  of  simulator 
training  is  quite  limited.  While  there  are  many  opinions  on  the  subject — 
almost  everyone  involved  in  activities  related  to  simulator  training  is  some- 
thing of  an  expert — there  was  little  agreement  among  the  Air  Force  personnel 
interviewed  during  the  project  as  to  how  simulators  should  be  designed  and 
used  for  training.  The  literature  reviewed  was  equally  inconclusive  in  most 
instances.  There  has  been  very  little  reported  research  on  training  simulator 
design  and  use,  and  that  which  has  been  reported  often  identifies  more  issues 
than  it  resolves.  An  obvious  conclusion  to  be  drawn  from  the  present  study 
is  that  too  little  attention  has  been  paid  to  date  to  the  effectiveness  of 
simulator  training. 
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APPENDIX 
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ON-SITE  SURVEY  OUTLINES 

A.  Simulator  Design  (Directed  to  simulator  operators  and  Instructors 
and  conducted  in  conjunction  with  an  inspection  of  the  simulator.) 

1.  Describe  motion 

2.  Describe  visual 

3.  Describe  lOS 

a.  location 

b.  principal  training  features 

c.  adequacy  for  intended  purpose 

d.  needs  for  redesign 

4.  Describe  trainee  positions 

5.  Pilot  comments 

a.  handling  characteristics 

b.  motion 

c.  visual 

d.  adequacy  of  malfunction  simulation 

e.  adequacy  of  environmental  simulation 

f.  needs  for  redesign 

g.  miscellaneous 

6.  Suggested  modifications 

B.  Simulator  Training  Effectiveness  (Directed  toward  personnel  responsible 
for  administration  and  management  of  simulator  training.) 

1.  Demonstrated  effectiveness 

a.  report  identification 

b.  design  model 

c.  describe  data  and  results 

d.  factors  Influencing  results 
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2.  Assumed  effectiveness 

a.  describe  perceptions 

b.  basis 

c.  documentation 

d.  factors  influencing  perception 

3.  Command  attitudes/perceptions 

4.  User  attitudes/perceptions 

5.  Factors  influencing  effectiveness 

C.  Trainees  (Directed  toward  personnel  undergoing  CCT  and  CT  at  the  time 
of  the  interview.) 

1.  Source  of  trainee  input 

2.  Typiccl  prior  operational  exi  trainees 

a.  hours  logged  by  A/C  types 

b.  aircraft  flown 

c.  type  of  unit 

3.  Prior  experience  with  simulators 

a.  prior  devices 

b.  recency  of  device  design 

c.  completeness  of  device  design 

d.  expressed  attitudes 

4.  Expressed  attitudes/expectations  concerning  present  simulator  training 

5.  Suggestions  for  change 

a.  simulator  design 

b.  simulator  use 

(1)  program  content,  sequence,  etc. 

(2)  simulator  training  schedule 

c.  other 
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6.  Perception  of  simulator  training 


a.  overall  value 

b.  specific  benefits/problems 

I 

D.  Instructors  (Directed  toward  personnel  serving  as  simulator  Instructors 
at  the  time  of  the  interview.) 

1.  Instructor  type  and  personal  data 

2.  Selection  prerequisite 

3.  Describe  simulator  instructor  training 

4.  Other  duties  and  time  requirements 

a.  flight  instructor 

b.  classroom  Instructor 

c.  administrative 

d.  miscellaneous 

5.  Typical  prior  operational  experience 

a.  hours  logged  in  aircraft  simulated 

b.  total  flight  time 

c.  types  of  units 

6.  Typical  prior  instructor  experience 

a.  simulator 

b.  other 

7.  Perception  of  their  assignment 

a.  effects  upon  career  advancement 

b.  value  of  experience  being  gained 

c.  personal  satisfaction 

8.  Factors  perceived  to  influence  Instructor  effectiveness 

i 

I 

i 
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